-- Logs begin at Fri 2022-07-01 15:53:11 UTC, end at Fri 2022-07-01 17:00:26 UTC. -- Jul 01 16:25:59.607166 dv-cinder-797608-41-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Jul 01 16:25:59.615966 dv-cinder-797608-41-iscsi devstack@keystone.service[80304]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Jul 01 16:25:59.618014 dv-cinder-797608-41-iscsi devstack@keystone.service[80304]: *** Starting uWSGI 2.0.18-debian (64bit) on [Fri Jul 1 16:25:59 2022] *** Jul 01 16:25:59.618014 dv-cinder-797608-41-iscsi devstack@keystone.service[80304]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Jul 01 16:25:59.618014 dv-cinder-797608-41-iscsi devstack@keystone.service[80304]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Jul 01 16:25:59.618014 dv-cinder-797608-41-iscsi devstack@keystone.service[80304]: nodename: dv-cinder-797608-41-iscsi Jul 01 16:25:59.618014 dv-cinder-797608-41-iscsi devstack@keystone.service[80304]: machine: x86_64 Jul 01 16:25:59.618014 dv-cinder-797608-41-iscsi devstack@keystone.service[80304]: clock source: unix Jul 01 16:25:59.618014 dv-cinder-797608-41-iscsi devstack@keystone.service[80304]: pcre jit disabled Jul 01 16:25:59.618014 dv-cinder-797608-41-iscsi devstack@keystone.service[80304]: detected number of CPU cores: 4 Jul 01 16:25:59.618014 dv-cinder-797608-41-iscsi devstack@keystone.service[80304]: current working directory: / Jul 01 16:25:59.618014 dv-cinder-797608-41-iscsi devstack@keystone.service[80304]: detected binary path: /usr/bin/uwsgi-core Jul 01 16:25:59.618014 dv-cinder-797608-41-iscsi devstack@keystone.service[80304]: your processes number limit is 47680 Jul 01 16:25:59.618014 dv-cinder-797608-41-iscsi devstack@keystone.service[80304]: your memory page size is 4096 bytes Jul 01 16:25:59.618784 dv-cinder-797608-41-iscsi devstack@keystone.service[80304]: detected max file descriptor number: 2048 Jul 01 16:25:59.618784 dv-cinder-797608-41-iscsi devstack@keystone.service[80304]: lock engine: pthread robust mutexes Jul 01 16:25:59.618784 dv-cinder-797608-41-iscsi devstack@keystone.service[80304]: thunder lock: enabled Jul 01 16:25:59.618784 dv-cinder-797608-41-iscsi devstack@keystone.service[80304]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Jul 01 16:25:59.618784 dv-cinder-797608-41-iscsi devstack@keystone.service[80304]: Python version: 3.8.10 (default, Mar 15 2022, 12:22:08) [GCC 9.4.0] Jul 01 16:25:59.664211 dv-cinder-797608-41-iscsi devstack@keystone.service[80304]: Python main interpreter initialized at 0x5637c3c4d9c0 Jul 01 16:25:59.664211 dv-cinder-797608-41-iscsi devstack@keystone.service[80304]: python threads support enabled Jul 01 16:25:59.664211 dv-cinder-797608-41-iscsi devstack@keystone.service[80304]: your server socket listen backlog is limited to 100 connections Jul 01 16:25:59.664211 dv-cinder-797608-41-iscsi devstack@keystone.service[80304]: your mercy for graceful operations on workers is 90 seconds Jul 01 16:25:59.664565 dv-cinder-797608-41-iscsi devstack@keystone.service[80304]: mapped 403077 bytes (393 KB) for 2 cores Jul 01 16:25:59.664641 dv-cinder-797608-41-iscsi devstack@keystone.service[80304]: *** Operational MODE: preforking *** Jul 01 16:25:59.664911 dv-cinder-797608-41-iscsi devstack@keystone.service[80304]: *** uWSGI is running in multiple interpreter mode *** Jul 01 16:25:59.664911 dv-cinder-797608-41-iscsi devstack@keystone.service[80304]: spawned uWSGI master process (pid: 80304) Jul 01 16:25:59.665133 dv-cinder-797608-41-iscsi systemd[1]: Started Devstack devstack@keystone.service. Jul 01 16:25:59.666255 dv-cinder-797608-41-iscsi devstack@keystone.service[80304]: spawned uWSGI worker 1 (pid: 80311, cores: 1) Jul 01 16:25:59.666255 dv-cinder-797608-41-iscsi devstack@keystone.service[80304]: spawned uWSGI worker 2 (pid: 80312, cores: 1) Jul 01 16:25:59.666255 dv-cinder-797608-41-iscsi devstack@keystone.service[80304]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Jul 01 16:26:05.162044 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 01 16:26:05.162044 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: from cryptography.utils import int_from_bytes Jul 01 16:26:05.162854 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 01 16:26:05.162854 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: from cryptography.utils import int_from_bytes Jul 01 16:26:05.213570 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jul 01 16:26:05.213756 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jul 01 16:26:05.213926 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] command line args: [] {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jul 01 16:26:05.214192 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jul 01 16:26:05.214393 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] ================================================================================ {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jul 01 16:26:05.214657 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] admin_token = **** {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 01 16:26:05.214907 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] config_dir = [] {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 01 16:26:05.215196 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 01 16:26:05.215353 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] config_source = [] {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 01 16:26:05.215499 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] debug = True {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 01 16:26:05.216030 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 01 16:26:05.216030 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 01 16:26:05.216856 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 01 16:26:05.216856 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 01 16:26:05.217194 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 01 16:26:05.217386 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] list_limit = None {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 01 16:26:05.217565 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] log_config_append = None {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 01 16:26:05.217751 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 01 16:26:05.217955 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] log_dir = None {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 01 16:26:05.218138 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] log_file = None {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 01 16:26:05.218404 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 01 16:26:05.218658 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 01 16:26:05.218909 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 01 16:26:05.219157 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 01 16:26:05.219335 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 01 16:26:05.219511 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 01 16:26:05.219757 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 01 16:26:05.220001 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 01 16:26:05.220234 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 01 16:26:05.220467 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 01 16:26:05.220705 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 01 16:26:05.220937 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 01 16:26:05.221185 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 01 16:26:05.221431 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 01 16:26:05.221708 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 01 16:26:05.222394 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.57/identity {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 01 16:26:05.222548 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] publish_errors = False {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 01 16:26:05.222808 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 01 16:26:05.223104 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 01 16:26:05.223286 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 01 16:26:05.223550 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 01 16:26:05.223792 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 01 16:26:05.224030 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] standard_threads = False {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 01 16:26:05.224131 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 01 16:26:05.224131 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: from cryptography.utils import int_from_bytes Jul 01 16:26:05.224643 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 01 16:26:05.224725 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 01 16:26:05.224725 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: from cryptography.utils import int_from_bytes Jul 01 16:26:05.224847 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 01 16:26:05.224847 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 01 16:26:05.224847 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] use_journal = False {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 01 16:26:05.225024 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] use_json = False {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 01 16:26:05.225142 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] use_stderr = False {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 01 16:26:05.225317 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] use_syslog = False {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 01 16:26:05.225548 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 01 16:26:05.225809 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.226047 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.226310 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.226583 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.226845 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.227084 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.227335 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.227574 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.227823 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.228072 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.228326 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.228560 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.228813 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.229049 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.229298 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.229547 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.229778 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.230036 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.230295 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.230571 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.230809 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.231064 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.231447 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.231707 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.232049 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.232782 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.233018 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.233309 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.233498 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.233747 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.234000 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.234232 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.234487 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.234740 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.235067 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.235867 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.235867 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.235867 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.236161 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.236161 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.236378 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.237033 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.237033 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.237033 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.237254 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.240613 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.240613 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.240613 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.240613 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.240613 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.240613 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.240613 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.240613 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.240613 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.241609 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.241609 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.241609 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.241609 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.241609 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.241609 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.241609 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.241609 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.241609 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.242245 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.242245 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.242245 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.242453 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.242689 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.243017 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.243186 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.244317 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.244317 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.244317 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.244317 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.244671 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.244671 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.244812 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.245043 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.245290 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.245527 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.245764 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.246002 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.246244 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.246505 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.246768 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.247010 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.247248 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.247486 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.248434 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.248795 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.249076 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.249319 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.249555 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.249790 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.250023 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.250261 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.250522 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.250766 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.251005 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.251258 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.251498 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.251758 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.252020 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.252262 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.252495 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] auth.external = None {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.252725 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.253022 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.253256 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.253486 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] auth.password = None {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.253717 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] auth.token = None {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.253956 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.254191 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.254460 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.254693 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.254941 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.255188 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.255419 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.255657 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] credential.caching = True {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.255906 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.256156 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.256405 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.256640 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.256884 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.257119 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.257388 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.257642 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.257879 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.258132 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.258455 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.258700 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.258963 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.259206 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.259463 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.259699 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] federation.caching = True {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.259934 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.260182 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.260432 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.260663 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.260970 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.261147 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.261401 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.261638 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.261900 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.262132 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.262398 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.262691 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] identity.caching = True {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.262937 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.263280 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.264018 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.264255 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.265430 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.265430 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.265430 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.265430 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.265873 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.265873 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.265990 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.266199 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.266467 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.266720 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.266969 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.267226 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.267473 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.267748 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.267988 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.268228 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.268477 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.268716 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.268948 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.269180 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.269420 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.269664 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.269917 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.270147 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.270409 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.270665 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.270902 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.271148 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.271396 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.271626 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.271864 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.272094 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.272331 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.272572 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.272818 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.273055 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.273299 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.273565 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.273806 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.274038 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.274269 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.274541 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.274792 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.274928 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80312) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jul 01 16:26:05.275058 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.275187 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80312) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jul 01 16:26:05.275317 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.275445 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG uwsgi [-] command line args: [] {{(pid=80312) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jul 01 16:26:05.275577 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.275710 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80312) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jul 01 16:26:05.275710 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG uwsgi [-] ================================================================================ {{(pid=80312) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jul 01 16:26:05.275950 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] ldap.user = None {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.275950 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.276203 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG uwsgi [-] admin_token = **** {{(pid=80312) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 01 16:26:05.276203 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG uwsgi [-] config_dir = [] {{(pid=80312) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 01 16:26:05.276457 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.276586 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80312) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 01 16:26:05.276713 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.276841 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG uwsgi [-] config_source = [] {{(pid=80312) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 01 16:26:05.276969 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.277102 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG uwsgi [-] debug = True {{(pid=80312) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 01 16:26:05.277242 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.277367 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80312) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 01 16:26:05.277539 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.277668 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80312) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 01 16:26:05.277668 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80312) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 01 16:26:05.277920 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.278049 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80312) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 01 16:26:05.278176 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.278303 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80312) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 01 16:26:05.278467 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.278596 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG uwsgi [-] list_limit = None {{(pid=80312) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 01 16:26:05.278737 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.278892 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG uwsgi [-] log_config_append = None {{(pid=80312) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 01 16:26:05.279020 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.279150 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80312) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 01 16:26:05.279283 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.279421 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG uwsgi [-] log_dir = None {{(pid=80312) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 01 16:26:05.279421 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG uwsgi [-] log_file = None {{(pid=80312) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 01 16:26:05.279628 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.279691 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80312) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 01 16:26:05.279875 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.279875 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.279875 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.280245 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80312) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 01 16:26:05.280378 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.280508 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80312) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 01 16:26:05.280636 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.280763 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80312) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 01 16:26:05.280889 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.281026 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80312) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 01 16:26:05.281026 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80312) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 01 16:26:05.281269 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.281392 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80312) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 01 16:26:05.281514 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.281647 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80312) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 01 16:26:05.281778 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.281900 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80312) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 01 16:26:05.282021 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.282154 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80312) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 01 16:26:05.282279 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.282378 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80312) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 01 16:26:05.282553 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.282687 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80312) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 01 16:26:05.282813 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.282939 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80312) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 01 16:26:05.283061 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.283185 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80312) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 01 16:26:05.283317 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.283449 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80312) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 01 16:26:05.283574 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.283574 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.283574 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.283928 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.57/identity {{(pid=80312) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 01 16:26:05.283928 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG uwsgi [-] publish_errors = False {{(pid=80312) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 01 16:26:05.284216 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.284355 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80312) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 01 16:26:05.284485 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.284632 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80312) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 01 16:26:05.284764 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.284902 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80312) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 01 16:26:05.285031 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.285158 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80312) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 01 16:26:05.285286 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.285413 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80312) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 01 16:26:05.285541 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] resource.caching = True {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.285676 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG uwsgi [-] standard_threads = False {{(pid=80312) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 01 16:26:05.285806 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.285939 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80312) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 01 16:26:05.286067 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.286202 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80312) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 01 16:26:05.286330 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.286487 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80312) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 01 16:26:05.286487 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG uwsgi [-] use_journal = False {{(pid=80312) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 01 16:26:05.286626 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.286705 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG uwsgi [-] use_json = False {{(pid=80312) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 01 16:26:05.286779 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.286852 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG uwsgi [-] use_stderr = False {{(pid=80312) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 01 16:26:05.286852 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG uwsgi [-] use_syslog = False {{(pid=80312) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 01 16:26:05.286997 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.287069 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80312) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 01 16:26:05.287142 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.287220 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80312) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.287299 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.287380 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80312) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.287455 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.287531 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80312) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.287604 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] role.caching = True {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.287676 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80312) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.287748 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] role.driver = sql {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.287829 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80312) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.287909 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.287982 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80312) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.288054 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.288126 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80312) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.288197 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.288277 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80312) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.288360 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.288434 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80312) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.288506 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.288578 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80312) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.288651 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.288731 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80312) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.288806 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.288878 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80312) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.288950 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.289034 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80312) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.289109 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.289255 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80312) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.289340 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.289516 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80312) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.289598 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.289768 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.289848 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80312) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.290003 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80312) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.290090 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.290244 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.290320 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80312) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.290528 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80312) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.290688 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.290781 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80312) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.290926 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.291008 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80312) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.291178 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.291259 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80312) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.291431 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.291548 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80312) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.291674 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.291760 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80312) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.291969 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.292092 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80312) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.292185 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80312) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.292262 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.292437 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80312) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.292578 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.292685 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80312) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.293248 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.293496 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80312) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.293496 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80312) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.293749 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.293877 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80312) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.293877 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80312) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.294121 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.294260 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80312) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.294260 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80312) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.294641 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.294850 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80312) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.295012 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.295178 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80312) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.295328 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.295328 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.295595 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80312) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.295726 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.295857 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80312) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.295987 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.296126 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80312) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.296260 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80312) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.296387 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.296519 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80312) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.296649 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80312) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.296777 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.296914 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80312) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.297045 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80312) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.297175 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] token.caching = True {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.297357 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.297483 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80312) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.297613 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.297749 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80312) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.297885 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80312) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.298013 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.298141 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80312) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.298268 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.298459 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80312) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.298596 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.298726 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80312) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.298855 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.298983 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80312) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.299111 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.299253 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80312) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.299386 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.299517 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80312) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.299648 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.299772 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80312) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.299897 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.300025 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80312) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.300163 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.300290 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80312) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.300416 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.300546 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80312) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.300676 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.300804 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80312) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.300948 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.301079 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80312) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.301207 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.301337 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80312) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.301462 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.301595 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80312) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.301725 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.301853 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80312) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.301982 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] database.connection = **** {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.302117 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80312) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.302259 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.302470 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80312) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.302598 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.302738 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80312) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.302878 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.303006 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80312) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.303134 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.303262 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80312) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.303390 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.303523 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80312) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.303648 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.303772 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80312) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.303897 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.304021 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80312) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.304143 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.304284 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80312) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.304409 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.304542 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80312) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.304671 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.304796 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80312) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.304920 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.305052 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80312) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.305178 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.305300 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80312) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.305423 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.305547 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80312) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.305675 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.305808 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80312) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.305934 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.306058 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80312) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.306187 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.306318 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80312) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.306478 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.306611 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80312) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.306736 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.306861 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80312) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.306983 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jul 01 16:26:05.307109 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80312) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.307109 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80312) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.307109 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80312) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.307109 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80312) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.307109 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80312) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.307109 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80312) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.307798 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80312) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.307798 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80312) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.307798 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80312) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.308164 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80312) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.308164 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80312) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.308394 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80312) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.308618 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80312) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.308851 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80312) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.309089 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80312) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.309319 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80312) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.309560 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80312) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.309798 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80312) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.310665 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80312) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.311483 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80312) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.311759 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80312) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.312027 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80312) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.312274 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80312) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.312509 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG uwsgi [-] auth.external = None {{(pid=80312) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.312742 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80312) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.313044 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80312) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.313288 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80312) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.313522 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG uwsgi [-] auth.password = None {{(pid=80312) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.314749 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG uwsgi [-] auth.token = None {{(pid=80312) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.314749 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80312) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.314749 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80312) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.314749 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80312) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.314749 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80312) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.315110 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80312) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.315184 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80312) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.315403 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80312) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.315642 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG uwsgi [-] credential.caching = True {{(pid=80312) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.315891 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80312) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.316142 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80312) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.316389 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80312) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.316628 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80312) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.316874 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80312) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.317110 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80312) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.317375 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80312) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.317629 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80312) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.317867 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80312) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.318121 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80312) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.318433 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80312) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.318682 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80312) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.318945 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80312) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.319189 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80312) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.320026 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80312) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.320026 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG uwsgi [-] federation.caching = True {{(pid=80312) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.320026 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80312) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.320721 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80312) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.320721 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80312) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.320721 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80312) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.321091 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80312) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.321091 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80312) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.321334 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80312) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.321520 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80312) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.321777 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80312) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.322012 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80312) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.322259 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80312) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.322520 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG uwsgi [-] identity.caching = True {{(pid=80312) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.322933 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80312) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.323078 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80312) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.323283 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80312) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.323467 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80312) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.323714 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80312) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.323946 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80312) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.324186 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80312) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.324437 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80312) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.324689 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80312) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.324923 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80312) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.325153 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80312) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.325384 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80312) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.325654 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80312) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.327260 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80312) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.327521 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80312) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.327789 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80312) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.328045 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80312) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.328336 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80312) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.328590 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80312) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.328837 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80312) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.329075 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80312) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.329317 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80312) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.329555 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80312) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.329797 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80312) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.330042 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80312) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.330290 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80312) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.330570 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80312) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.330805 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80312) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.331059 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80312) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.331310 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80312) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.331545 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80312) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.331795 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80312) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.332287 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80312) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.332528 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80312) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.332767 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80312) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.332998 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80312) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.333236 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80312) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.333474 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80312) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.333727 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80312) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.333966 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80312) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.334204 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80312) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.334484 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80312) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.334736 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80312) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.334968 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80312) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.335198 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80312) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.335446 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80312) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.335696 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80312) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.335933 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80312) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.336168 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80312) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.336404 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80312) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.336635 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG uwsgi [-] ldap.user = None {{(pid=80312) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.336874 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80312) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.337134 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80312) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.337374 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80312) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.337624 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80312) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.337872 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80312) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.338121 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80312) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.338247 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80311) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 01 16:26:05.338400 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80311) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 01 16:26:05.338527 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80312) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.338653 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80311) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 01 16:26:05.338792 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80312) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.338925 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80312) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.339065 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80312) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.339266 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80312) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.339497 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80312) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.339908 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80312) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.340173 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80312) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.340426 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80312) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.340678 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80312) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.340938 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80312) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.341612 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80312) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.341612 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80312) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.342259 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80312) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.342543 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80312) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.342788 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80312) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.343031 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80312) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.343283 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80312) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.343538 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80312) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.343780 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80312) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.344044 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80312) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.344277 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80312) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.344525 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80312) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.344763 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80312) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.345029 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80312) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.345240 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80312) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.345491 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80312) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.345734 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80312) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.345968 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80312) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.346206 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80312) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.346478 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG uwsgi [-] resource.caching = True {{(pid=80312) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.346732 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80312) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.346985 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80312) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.347224 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80312) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.347349 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80311) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 01 16:26:05.347483 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80312) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.347717 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80312) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.347955 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80312) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.348203 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80312) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.348440 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80312) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.348681 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80312) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.348917 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG uwsgi [-] role.caching = True {{(pid=80312) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.349164 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG uwsgi [-] role.driver = sql {{(pid=80312) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.349395 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80312) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.349646 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80312) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.349895 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80312) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.350142 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80312) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.350412 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80312) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.350664 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80312) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.350925 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80312) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.351165 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80312) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.351416 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80312) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.351648 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80312) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.351898 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80312) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.352144 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80312) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.352414 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80312) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.352641 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80312) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.353072 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80312) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.353307 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80312) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.353560 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80312) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.353815 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80312) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.354065 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80312) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.354316 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80312) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.354595 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80312) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.354853 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80312) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.355105 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80312) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.355346 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80312) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.355577 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80312) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.355810 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80312) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.356041 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80312) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.356292 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80312) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.356548 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80312) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.356793 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80312) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.357066 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80312) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.357927 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80312) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.358313 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80312) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.358435 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG uwsgi [-] token.caching = True {{(pid=80312) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.358662 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80312) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.358917 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80312) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.359159 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80312) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.359418 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80312) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.359666 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80312) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.359905 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80312) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.360152 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80312) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.360397 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80312) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.360644 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80312) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.360880 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80312) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.361119 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80312) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.361355 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80312) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.361602 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80312) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.361851 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80312) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.362080 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80312) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.362326 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80312) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.362653 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80312) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.362916 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG uwsgi [-] database.connection = **** {{(pid=80312) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.363164 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80312) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.363416 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80312) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.363654 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80312) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.363903 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80312) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.364145 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80312) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.364384 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80312) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.364620 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80312) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.364856 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80312) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.365130 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80312) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.365341 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80312) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.365585 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80312) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.365820 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80312) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.366068 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80312) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.366303 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80312) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.366573 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80312) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.366804 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80312) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.367043 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80312) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.367277 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80312) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 16:26:05.367445 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80312) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jul 01 16:26:05.376166 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80311) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 01 16:26:05.376479 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80311) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 01 16:26:05.396814 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80312) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 01 16:26:05.397080 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80312) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 01 16:26:05.397249 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80312) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 01 16:26:05.406665 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80312) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 01 16:26:05.425650 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80312) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 01 16:26:05.425907 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80312) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 01 16:26:05.521284 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80311) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 01 16:26:05.563708 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80312) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 01 16:26:05.573428 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80311) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 01 16:26:05.573673 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80311) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 01 16:26:05.600726 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80312) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 01 16:26:05.601002 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80312) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 01 16:26:05.640754 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80311) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 01 16:26:05.641018 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80311) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 01 16:26:05.641185 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80311) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 01 16:26:05.641404 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80311) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 01 16:26:05.641624 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80311) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 01 16:26:05.641841 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80311) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 01 16:26:05.643092 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80312) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 01 16:26:05.643329 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80312) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 01 16:26:05.643553 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80312) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 01 16:26:05.643775 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80312) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 01 16:26:05.643995 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80312) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 01 16:26:05.644212 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80312) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 01 16:26:05.667825 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80311) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 01 16:26:05.668114 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80311) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 01 16:26:05.668253 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80311) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 01 16:26:05.668464 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80311) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 01 16:26:05.668684 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80311) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 01 16:26:05.668898 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80311) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 01 16:26:05.669028 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80312) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 01 16:26:05.669174 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80311) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 01 16:26:05.669303 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80312) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 01 16:26:05.669523 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80312) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 01 16:26:05.669754 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80312) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 01 16:26:05.669976 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80312) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 01 16:26:05.670192 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80312) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 01 16:26:05.670679 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80312) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 01 16:26:05.680672 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80311) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 01 16:26:05.681089 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80312) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 01 16:26:05.693690 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80311) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 01 16:26:05.694028 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80312) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 01 16:26:05.694239 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80311) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 01 16:26:05.694423 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80312) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 01 16:26:05.694539 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80311) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 01 16:26:05.694820 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80312) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 01 16:26:05.695040 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80311) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 01 16:26:05.695040 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80311) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 01 16:26:05.695417 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80312) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 01 16:26:05.695638 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80311) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 01 16:26:05.695839 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80312) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 01 16:26:05.696042 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80311) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 01 16:26:05.696259 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80312) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 01 16:26:05.696259 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80312) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 01 16:26:05.713453 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80311) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 16:26:05.713699 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80312) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 16:26:05.713795 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80311) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 01 16:26:05.713887 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80312) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 01 16:26:05.713985 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80311) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 16:26:05.714061 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80312) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 16:26:05.714183 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80311) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 01 16:26:05.714443 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80312) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 01 16:26:05.714574 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80311) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 16:26:05.714718 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80312) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 16:26:05.714845 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80311) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 16:26:05.714971 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80312) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 16:26:05.715094 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80311) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 16:26:05.715220 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80312) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 16:26:05.715365 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80311) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 16:26:05.715490 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80312) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 16:26:05.727115 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80311) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 16:26:05.727331 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80312) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 16:26:05.727485 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80311) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 16:26:05.727618 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80312) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 16:26:05.727766 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80311) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 16:26:05.727939 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80312) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 16:26:05.728064 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80311) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 16:26:05.728144 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80312) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 16:26:05.736382 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80312) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 01 16:26:05.736529 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80311) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 01 16:26:05.739886 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80312) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 01 16:26:05.740009 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80311) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 01 16:26:05.740162 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80312) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 16:26:05.740250 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80311) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 16:26:05.740446 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80312) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 16:26:05.740532 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80311) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 16:26:05.740732 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80312) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 16:26:05.740814 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80311) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 16:26:05.741009 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80311) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 16:26:05.741090 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80312) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 16:26:05.741268 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80311) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 16:26:05.741386 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80312) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 16:26:05.741535 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80311) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 16:26:05.741661 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80312) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 16:26:05.741804 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80311) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 16:26:05.741938 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80312) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 16:26:05.742068 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80311) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 16:26:05.742215 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80312) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 16:26:05.742335 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80311) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 16:26:05.742521 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80312) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 16:26:05.742616 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80311) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 16:26:05.742797 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80312) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 16:26:05.769908 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80311) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 16:26:05.770105 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80312) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 16:26:05.771576 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80311) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 01 16:26:05.771745 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80312) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 01 16:26:05.771861 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80311) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 16:26:05.772025 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80312) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 16:26:05.777133 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80311) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 01 16:26:05.777294 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80312) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 01 16:26:05.777389 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80311) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 16:26:05.777572 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80312) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 16:26:05.777792 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80311) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 16:26:05.777891 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80312) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 16:26:05.785693 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80311) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 01 16:26:05.785864 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80312) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 01 16:26:05.786025 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80311) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 16:26:05.786145 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80312) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 16:26:05.790453 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80311) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 01 16:26:05.790554 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80312) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 01 16:26:05.790737 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80311) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 16:26:05.790922 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80312) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 16:26:05.791022 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80311) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 16:26:05.791207 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80312) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 16:26:05.791289 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80311) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 16:26:05.791496 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80312) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 16:26:05.791578 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80311) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 16:26:05.791777 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80312) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 16:26:05.791861 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80311) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 16:26:05.792056 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80312) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 16:26:05.792138 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80311) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 16:26:05.792320 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80311) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 16:26:05.792408 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80312) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 16:26:05.792591 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80312) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 16:26:05.811703 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80312) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 16:26:05.811991 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80312) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 16:26:05.812083 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80311) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 16:26:05.812315 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80311) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 16:26:05.816950 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80312) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 01 16:26:05.817125 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80311) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 01 16:26:05.820806 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80311) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 16:26:05.820916 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80312) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 16:26:05.821097 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80311) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 16:26:05.821202 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80312) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 16:26:05.826872 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80311) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 01 16:26:05.827008 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80312) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 01 16:26:05.830418 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80311) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 01 16:26:05.830539 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80312) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 01 16:26:05.834375 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80311) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 16:26:05.834628 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80312) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 16:26:05.834729 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80311) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 16:26:05.834947 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80312) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 16:26:05.835087 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80311) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 16:26:05.835232 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80312) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 16:26:05.835399 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80311) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 16:26:05.835597 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80312) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 16:26:05.835686 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80311) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 16:26:05.835839 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80312) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 16:26:05.835967 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80311) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 16:26:05.836130 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80312) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 16:26:05.856195 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80312) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 01 16:26:05.856459 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80311) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 01 16:26:05.856586 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80312) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 16:26:05.856752 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80312) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 16:26:05.856992 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80311) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 16:26:05.857178 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80312) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 16:26:05.857447 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80311) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 16:26:05.857886 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80311) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 16:26:05.865642 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80312) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 16:26:05.867397 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80312) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 16:26:05.867670 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80312) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 16:26:05.869473 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80311) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 16:26:05.870825 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80312) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 01 16:26:05.871106 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80312) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 16:26:05.871252 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80311) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 16:26:05.871395 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80312) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 16:26:05.871526 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80311) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 16:26:05.871718 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80312) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 16:26:05.871961 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80312) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 16:26:05.875051 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80311) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 01 16:26:05.875332 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80311) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 16:26:05.875610 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80311) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 16:26:05.875890 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80311) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 16:26:05.876176 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80311) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 16:26:05.885537 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80312) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 01 16:26:05.885818 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80312) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 16:26:05.886102 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80312) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 16:26:05.886403 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80312) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 16:26:05.886692 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80312) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 16:26:05.886967 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80312) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 16:26:05.887253 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80312) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 16:26:05.890072 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80311) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 01 16:26:05.890390 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80311) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 16:26:05.890663 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80311) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 16:26:05.890943 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80311) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 16:26:05.891220 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80311) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 16:26:05.892938 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80311) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 16:26:05.893243 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80311) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 16:26:05.907900 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80312) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 01 16:26:05.911317 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80312) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 01 16:26:05.912084 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80311) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 01 16:26:05.914659 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80312) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 16:26:05.915409 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80311) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 01 16:26:05.916277 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80312) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 16:26:05.917930 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80312) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 01 16:26:05.918519 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80312) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 16:26:05.918519 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80312) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 16:26:05.918801 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80311) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 16:26:05.921093 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80311) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 16:26:05.922757 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80311) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 01 16:26:05.923032 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80311) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 16:26:05.923318 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80311) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 16:26:05.926499 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80312) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 16:26:05.928135 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80312) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 01 16:26:05.930618 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80311) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 16:26:05.931540 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80312) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 16:26:05.931795 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80312) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 16:26:05.932169 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80312) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 16:26:05.932295 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80311) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 01 16:26:05.932465 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80312) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 16:26:05.936213 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80311) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 16:26:05.936505 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80311) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 16:26:05.936904 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80311) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 16:26:05.937189 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80311) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 16:26:05.942248 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80312) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 01 16:26:05.942556 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80312) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 16:26:05.942962 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80312) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 16:26:05.946284 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80311) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 01 16:26:05.946588 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80311) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 16:26:05.946870 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80311) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 16:26:05.950954 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80312) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 01 16:26:05.951241 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80312) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 16:26:05.951521 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80312) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 16:26:05.951794 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80312) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 16:26:05.952062 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80312) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 16:26:05.952329 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80312) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 16:26:05.952608 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80312) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 16:26:05.952872 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80312) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 16:26:05.953151 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80312) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 16:26:05.953424 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80312) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 16:26:05.953705 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80312) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 16:26:05.953966 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80312) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 16:26:05.954233 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80312) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 16:26:05.954519 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80312) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 16:26:05.954790 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80311) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 01 16:26:05.955125 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80311) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 16:26:05.955342 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80311) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 16:26:05.955616 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80311) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 16:26:05.955887 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80311) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 16:26:05.956266 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80311) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 16:26:05.956551 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80311) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 16:26:05.956815 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80311) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 16:26:05.957353 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80311) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 16:26:05.957695 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80311) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 16:26:05.957979 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80311) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 16:26:05.958244 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80311) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 16:26:05.958535 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80311) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 16:26:05.958819 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80311) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 16:26:05.994754 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x5637c3c4d9c0 pid: 80312 (default app) Jul 01 16:26:05.997429 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x5637c3c4d9c0 pid: 80311 (default app) Jul 01 16:26:05.998226 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299daa55-0c7c-4f09-91e3-fdcd992a860c None None] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:26:05.998553 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299daa55-0c7c-4f09-91e3-fdcd992a860c None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:26:05.998840 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299daa55-0c7c-4f09-91e3-fdcd992a860c None None] PATH_INFO: `/v3` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:26:06.003176 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1/1] 10.222.0.57 () {50 vars in 791 bytes} [Fri Jul 1 16:26:05 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:26:13.705770 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba10d0a-973b-438d-8391-29828073da1d None None] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:26:13.706369 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba10d0a-973b-438d-8391-29828073da1d None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:26:13.706369 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba10d0a-973b-438d-8391-29828073da1d None None] PATH_INFO: `/` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:26:13.710928 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1/2] 10.222.0.57 () {54 vars in 926 bytes} [Fri Jul 1 16:26:13 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 01 16:26:13.716977 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbcd9eb4-0867-41a5-baf6-5a4d91c85cbb None None] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:26:13.717778 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbcd9eb4-0867-41a5-baf6-5a4d91c85cbb None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:26:13.717778 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbcd9eb4-0867-41a5-baf6-5a4d91c85cbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:26:13.787498 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: INFO dbcounter [None req-cbcd9eb4-0867-41a5-baf6-5a4d91c85cbb None None] Registered counter for database keystone Jul 01 16:26:13.809170 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG oslo_db.sqlalchemy.engines [None req-cbcd9eb4-0867-41a5-baf6-5a4d91c85cbb None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80312) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jul 01 16:26:13.816231 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writer thread running {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jul 01 16:26:14.008412 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG passlib.handlers.bcrypt [None req-cbcd9eb4-0867-41a5-baf6-5a4d91c85cbb None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80312) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jul 01 16:26:14.008755 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG passlib.handlers.bcrypt [None req-cbcd9eb4-0867-41a5-baf6-5a4d91c85cbb None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80312) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jul 01 16:26:14.063794 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.auth.core [None req-cbcd9eb4-0867-41a5-baf6-5a4d91c85cbb None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80312) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:26:14.065425 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-cbcd9eb4-0867-41a5-baf6-5a4d91c85cbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:26:14.187146 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 2/3] 10.222.0.57 () {58 vars in 1023 bytes} [Fri Jul 1 16:26:13 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 471 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Jul 01 16:26:14.358121 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba10d0a-973b-438d-8391-29828073da1d None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:26:14.428398 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: INFO dbcounter [None req-3ba10d0a-973b-438d-8391-29828073da1d None None] Registered counter for database keystone Jul 01 16:26:14.448011 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG oslo_db.sqlalchemy.engines [None req-3ba10d0a-973b-438d-8391-29828073da1d None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80311) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jul 01 16:26:14.454463 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writer thread running {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jul 01 16:26:14.570656 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba10d0a-973b-438d-8391-29828073da1d None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:26:14.574849 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4722921-310d-4d4a-b25f-a5bde39e970a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:26:14.576131 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4722921-310d-4d4a-b25f-a5bde39e970a admin admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:26:14.576409 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4722921-310d-4d4a-b25f-a5bde39e970a admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:26:14.576681 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4722921-310d-4d4a-b25f-a5bde39e970a admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:26:14.584994 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4722921-310d-4d4a-b25f-a5bde39e970a admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:26:14.636403 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: WARNING keystone.common.rbac_enforcer.enforcer [None req-c4722921-310d-4d4a-b25f-a5bde39e970a admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jul 01 16:26:14.636682 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4722921-310d-4d4a-b25f-a5bde39e970a admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:26:14.646845 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 2/4] 10.222.0.57 () {58 vars in 1324 bytes} [Fri Jul 1 16:26:14 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 289 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 01 16:26:14.652828 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbcd9eb4-0867-41a5-baf6-5a4d91c85cbb None None] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:26:14.667894 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbcd9eb4-0867-41a5-baf6-5a4d91c85cbb None None] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:26:14.672474 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece4840d-03ea-4d20-899e-06b71b4912f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:26:14.673589 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece4840d-03ea-4d20-899e-06b71b4912f1 admin admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:26:14.673882 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece4840d-03ea-4d20-899e-06b71b4912f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:26:14.674161 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece4840d-03ea-4d20-899e-06b71b4912f1 admin admin] PATH_INFO: `/v3/projects/62a34c02685249b399e8facc39cc11f8` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:26:14.675704 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece4840d-03ea-4d20-899e-06b71b4912f1 admin admin] RBAC: Authorizing `identity:get_project(project_id=62a34c02685249b399e8facc39cc11f8)` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:26:14.729440 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: WARNING keystone.common.rbac_enforcer.enforcer [None req-ece4840d-03ea-4d20-899e-06b71b4912f1 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jul 01 16:26:14.730006 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece4840d-03ea-4d20-899e-06b71b4912f1 admin admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:26:14.734485 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 3/5] 10.222.0.57 () {56 vars in 1207 bytes} [Fri Jul 1 16:26:14 2022] GET /identity/v3/projects/62a34c02685249b399e8facc39cc11f8 => generated 344 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 01 16:26:16.816712 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38abb207-ab3c-4eab-919b-8419889e89aa None None] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:26:16.817253 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38abb207-ab3c-4eab-919b-8419889e89aa None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:26:16.817253 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38abb207-ab3c-4eab-919b-8419889e89aa None None] PATH_INFO: `/` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:26:16.821510 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 3/6] 10.222.0.57 () {54 vars in 926 bytes} [Fri Jul 1 16:26:16 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 01 16:26:16.827331 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf017a70-f555-4c93-be71-788cabf11a86 None None] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:26:16.827725 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf017a70-f555-4c93-be71-788cabf11a86 None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:26:16.827888 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf017a70-f555-4c93-be71-788cabf11a86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:26:16.847423 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.auth.core [None req-cf017a70-f555-4c93-be71-788cabf11a86 None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80312) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:26:16.848953 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-cf017a70-f555-4c93-be71-788cabf11a86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:26:16.858584 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 4/7] 10.222.0.57 () {58 vars in 1023 bytes} [Fri Jul 1 16:26:16 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Jul 01 16:26:16.881805 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38abb207-ab3c-4eab-919b-8419889e89aa None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:26:16.894025 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38abb207-ab3c-4eab-919b-8419889e89aa None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:26:16.898175 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbcbad4b-0ee0-44af-b571-8a530d6cc0b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:26:16.898838 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcbad4b-0ee0-44af-b571-8a530d6cc0b1 admin admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:26:16.899125 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcbad4b-0ee0-44af-b571-8a530d6cc0b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:26:16.899402 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcbad4b-0ee0-44af-b571-8a530d6cc0b1 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:26:16.907144 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbcbad4b-0ee0-44af-b571-8a530d6cc0b1 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:26:16.908906 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbcbad4b-0ee0-44af-b571-8a530d6cc0b1 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:26:16.917509 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 4/8] 10.222.0.57 () {58 vars in 1324 bytes} [Fri Jul 1 16:26:16 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 01 16:26:16.923424 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf017a70-f555-4c93-be71-788cabf11a86 None None] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:26:16.932434 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf017a70-f555-4c93-be71-788cabf11a86 None None] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:26:16.937780 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6199c69-0bd3-4034-80c8-110d878235df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:26:16.938693 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6199c69-0bd3-4034-80c8-110d878235df admin admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:26:16.938983 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6199c69-0bd3-4034-80c8-110d878235df admin admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:26:16.939264 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6199c69-0bd3-4034-80c8-110d878235df admin admin] PATH_INFO: `/v3/users/9996f66c673f4673ab2342b66da9acc2` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:26:16.940648 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6199c69-0bd3-4034-80c8-110d878235df admin admin] RBAC: Authorizing `identity:get_user(user_id=9996f66c673f4673ab2342b66da9acc2)` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:26:16.942510 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6199c69-0bd3-4034-80c8-110d878235df admin admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:26:16.944082 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 5/9] 10.222.0.57 () {56 vars in 1198 bytes} [Fri Jul 1 16:26:16 2022] GET /identity/v3/users/9996f66c673f4673ab2342b66da9acc2 => generated 247 bytes in 21 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:26:19.054046 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fdc0ec-9c0c-425e-8584-cffad0b76bd2 None None] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:26:19.054612 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fdc0ec-9c0c-425e-8584-cffad0b76bd2 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:26:19.054612 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fdc0ec-9c0c-425e-8584-cffad0b76bd2 None None] PATH_INFO: `/` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:26:19.055983 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 5/10] 10.222.0.57 () {54 vars in 926 bytes} [Fri Jul 1 16:26:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 16:26:19.061057 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e020ebb-1e69-46c5-be7c-048719f73430 None None] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:26:19.061455 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e020ebb-1e69-46c5-be7c-048719f73430 None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:26:19.061622 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e020ebb-1e69-46c5-be7c-048719f73430 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:26:19.079118 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.auth.core [None req-1e020ebb-1e69-46c5-be7c-048719f73430 None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80312) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:26:19.080318 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-1e020ebb-1e69-46c5-be7c-048719f73430 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:26:19.162539 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8662dd6d-b75a-4c48-9b89-6547db68a16a None None] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:26:19.163740 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8662dd6d-b75a-4c48-9b89-6547db68a16a None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:26:19.163872 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 6/11] 10.222.0.57 () {58 vars in 1023 bytes} [Fri Jul 1 16:26:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 01 16:26:19.163962 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8662dd6d-b75a-4c48-9b89-6547db68a16a None None] PATH_INFO: `/` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:26:19.164361 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 6/12] 10.222.0.57 () {54 vars in 926 bytes} [Fri Jul 1 16:26:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 01 16:26:19.171620 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a159527d-9bfd-4571-8c94-ea7a80dd89c0 None None] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:26:19.171955 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a159527d-9bfd-4571-8c94-ea7a80dd89c0 None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:26:19.172243 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a159527d-9bfd-4571-8c94-ea7a80dd89c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:26:19.187147 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8662dd6d-b75a-4c48-9b89-6547db68a16a None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:26:19.189569 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.auth.core [None req-a159527d-9bfd-4571-8c94-ea7a80dd89c0 None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80312) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:26:19.190813 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-a159527d-9bfd-4571-8c94-ea7a80dd89c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:26:19.208915 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=100 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:26:19.265042 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 7/13] 10.222.0.57 () {58 vars in 1023 bytes} [Fri Jul 1 16:26:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 01 16:26:19.287853 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a159527d-9bfd-4571-8c94-ea7a80dd89c0 None None] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:26:19.337014 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8662dd6d-b75a-4c48-9b89-6547db68a16a None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:26:19.370937 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da1e664-7b9f-4864-9abb-bb3dc7633b55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:26:19.372419 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da1e664-7b9f-4864-9abb-bb3dc7633b55 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:26:19.372800 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da1e664-7b9f-4864-9abb-bb3dc7633b55 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:26:19.373025 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da1e664-7b9f-4864-9abb-bb3dc7633b55 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:26:19.384367 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da1e664-7b9f-4864-9abb-bb3dc7633b55 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 01 16:26:19.391144 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a159527d-9bfd-4571-8c94-ea7a80dd89c0 None None] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:26:19.395815 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application [None req-9da1e664-7b9f-4864-9abb-bb3dc7633b55 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 01 16:26:19.395815 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 16:26:19.395815 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 16:26:19.395815 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 16:26:19.395815 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 16:26:19.395815 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 16:26:19.395815 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 16:26:19.395815 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 16:26:19.395815 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 16:26:19.395815 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 16:26:19.395815 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 16:26:19.395815 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 16:26:19.395815 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 01 16:26:19.395815 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 01 16:26:19.395815 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 01 16:26:19.395815 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 01 16:26:19.396898 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 01 16:26:19.396898 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 01 16:26:19.396898 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 01 16:26:19.396898 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 01 16:26:19.396898 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 16:26:19.396898 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 16:26:19.396898 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 16:26:19.396898 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 16:26:19.396898 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 16:26:19.396898 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 16:26:19.396898 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 16:26:19.396898 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application with Lock( Jul 01 16:26:19.396898 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 16:26:19.396898 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._enter() Jul 01 16:26:19.396898 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 16:26:19.396898 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 16:26:19.397908 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 16:26:19.397908 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.creator() Jul 01 16:26:19.397908 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 16:26:19.397908 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application created_value = creator( Jul 01 16:26:19.397908 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 01 16:26:19.397908 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 01 16:26:19.397908 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 01 16:26:19.397908 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 01 16:26:19.397908 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 01 16:26:19.397908 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 01 16:26:19.397908 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 01 16:26:19.397908 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Jul 01 16:26:19.397908 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 7/14] 10.222.0.57 () {56 vars in 1096 bytes} [Fri Jul 1 16:26:19 2022] GET /identity/v3/roles/admin => generated 83 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 01 16:26:19.403222 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da1e664-7b9f-4864-9abb-bb3dc7633b55 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:26:19.426141 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa790e7-fd68-4444-9882-0bafbce9b599 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:26:19.426742 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa790e7-fd68-4444-9882-0bafbce9b599 None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:26:19.427034 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa790e7-fd68-4444-9882-0bafbce9b599 None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:26:19.427310 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa790e7-fd68-4444-9882-0bafbce9b599 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:26:19.439753 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:26:19.449844 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=100 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:26:19.499685 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa790e7-fd68-4444-9882-0bafbce9b599 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:26:19.501164 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa790e7-fd68-4444-9882-0bafbce9b599 None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:26:19.510133 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da1e664-7b9f-4864-9abb-bb3dc7633b55 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:26:19.545703 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3a8d6b-9040-4e93-be5d-0fd9b9639d4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:26:19.546394 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3a8d6b-9040-4e93-be5d-0fd9b9639d4a None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:26:19.546684 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3a8d6b-9040-4e93-be5d-0fd9b9639d4a None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:26:19.546967 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3a8d6b-9040-4e93-be5d-0fd9b9639d4a None admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:26:19.547728 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd3a8d6b-9040-4e93-be5d-0fd9b9639d4a None admin] RBAC: Adding query filter params (name=admin) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:26:19.548038 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd3a8d6b-9040-4e93-be5d-0fd9b9639d4a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:26:19.549443 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd3a8d6b-9040-4e93-be5d-0fd9b9639d4a None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:26:19.561540 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 8/15] 10.222.0.57 () {56 vars in 1099 bytes} [Fri Jul 1 16:26:19 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 01 16:26:19.567136 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3a8d6b-9040-4e93-be5d-0fd9b9639d4a None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:26:19.602930 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 8/16] 10.222.0.57 () {58 vars in 1282 bytes} [Fri Jul 1 16:26:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 01 16:26:19.609068 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa790e7-fd68-4444-9882-0bafbce9b599 None admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:26:19.660250 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=100 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:26:19.663723 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:26:19.671513 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3a8d6b-9040-4e93-be5d-0fd9b9639d4a None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:26:19.705951 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27ac855-e7dd-47cd-bced-57190b9c06d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:26:19.707033 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27ac855-e7dd-47cd-bced-57190b9c06d9 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:26:19.707325 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27ac855-e7dd-47cd-bced-57190b9c06d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:26:19.707605 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27ac855-e7dd-47cd-bced-57190b9c06d9 None admin] PATH_INFO: `/v3/users/9996f66c673f4673ab2342b66da9acc2` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:26:19.708632 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa790e7-fd68-4444-9882-0bafbce9b599 None admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:26:19.709247 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b27ac855-e7dd-47cd-bced-57190b9c06d9 None admin] RBAC: Authorizing `identity:get_user(user_id=9996f66c673f4673ab2342b66da9acc2)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:26:19.710741 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b27ac855-e7dd-47cd-bced-57190b9c06d9 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:26:19.712399 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 9/17] 10.222.0.57 () {56 vars in 1177 bytes} [Fri Jul 1 16:26:19 2022] GET /identity/v3/users/9996f66c673f4673ab2342b66da9acc2 => generated 247 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 01 16:26:19.718510 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27ac855-e7dd-47cd-bced-57190b9c06d9 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:26:19.744177 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661336a8-dd62-4c11-8c52-876160cdc386 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:26:19.744974 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661336a8-dd62-4c11-8c52-876160cdc386 None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:26:19.745260 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661336a8-dd62-4c11-8c52-876160cdc386 None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:26:19.745544 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661336a8-dd62-4c11-8c52-876160cdc386 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:26:19.755879 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-661336a8-dd62-4c11-8c52-876160cdc386 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:26:19.757388 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-661336a8-dd62-4c11-8c52-876160cdc386 None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:26:19.782243 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application [None req-661336a8-dd62-4c11-8c52-876160cdc386 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 16:26:19.782243 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 16:26:19.782243 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 16:26:19.782243 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 16:26:19.782243 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 16:26:19.782243 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 16:26:19.782243 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 16:26:19.782243 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 16:26:19.782243 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 16:26:19.782243 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 16:26:19.782243 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 16:26:19.782243 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 16:26:19.782243 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 01 16:26:19.782243 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 01 16:26:19.782243 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 01 16:26:19.783326 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 01 16:26:19.783326 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 16:26:19.783326 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 16:26:19.783326 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 16:26:19.783326 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 16:26:19.783326 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 16:26:19.783326 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 16:26:19.783326 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 16:26:19.783326 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application with Lock( Jul 01 16:26:19.783326 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 16:26:19.783326 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self._enter() Jul 01 16:26:19.783326 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 16:26:19.783326 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 16:26:19.783326 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 16:26:19.783326 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.creator() Jul 01 16:26:19.783326 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 16:26:19.783326 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application created_value = creator( Jul 01 16:26:19.784331 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 01 16:26:19.784331 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 01 16:26:19.784331 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 16:26:19.784331 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application Jul 01 16:26:19.784331 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 9/18] 10.222.0.57 () {56 vars in 1108 bytes} [Fri Jul 1 16:26:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 01 16:26:19.789517 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661336a8-dd62-4c11-8c52-876160cdc386 None admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:26:19.829157 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27ac855-e7dd-47cd-bced-57190b9c06d9 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:26:19.864175 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452bdac9-52cc-4f82-9cc7-2d2a5aff7353 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:26:19.864970 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452bdac9-52cc-4f82-9cc7-2d2a5aff7353 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:26:19.865259 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452bdac9-52cc-4f82-9cc7-2d2a5aff7353 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:26:19.865541 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452bdac9-52cc-4f82-9cc7-2d2a5aff7353 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:26:19.866865 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-452bdac9-52cc-4f82-9cc7-2d2a5aff7353 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:26:19.868346 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-452bdac9-52cc-4f82-9cc7-2d2a5aff7353 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:26:19.869946 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 10/19] 10.222.0.57 () {56 vars in 1108 bytes} [Fri Jul 1 16:26:19 2022] GET /identity/v3/domains/default => generated 199 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 01 16:26:19.876173 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452bdac9-52cc-4f82-9cc7-2d2a5aff7353 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:26:19.898000 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661336a8-dd62-4c11-8c52-876160cdc386 None admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:26:19.900647 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=100 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:26:19.914309 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:26:19.935816 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e480759d-da56-4b31-89b4-e8aa50bba32b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:26:19.936493 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e480759d-da56-4b31-89b4-e8aa50bba32b None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:26:19.936779 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e480759d-da56-4b31-89b4-e8aa50bba32b None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:26:19.937069 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e480759d-da56-4b31-89b4-e8aa50bba32b None admin] PATH_INFO: `/v3/domains` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:26:19.937808 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e480759d-da56-4b31-89b4-e8aa50bba32b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80312) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:26:19.938116 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e480759d-da56-4b31-89b4-e8aa50bba32b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:26:19.939548 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e480759d-da56-4b31-89b4-e8aa50bba32b None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:26:19.957966 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 10/20] 10.222.0.57 () {56 vars in 1109 bytes} [Fri Jul 1 16:26:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 01 16:26:20.000074 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452bdac9-52cc-4f82-9cc7-2d2a5aff7353 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:26:20.042259 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1c791f-a5e8-48da-8a21-44b3efc5132d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:26:20.042990 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1c791f-a5e8-48da-8a21-44b3efc5132d None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:26:20.043532 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1c791f-a5e8-48da-8a21-44b3efc5132d None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:26:20.044078 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1c791f-a5e8-48da-8a21-44b3efc5132d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:26:20.045683 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1c791f-a5e8-48da-8a21-44b3efc5132d None admin] RBAC: Adding query filter params (role.id=84ee739146934c1ab32c501ff8093644, scope.domain.id=default, user.id=9996f66c673f4673ab2342b66da9acc2) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:26:20.046255 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1c791f-a5e8-48da-8a21-44b3efc5132d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:26:20.048039 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1c791f-a5e8-48da-8a21-44b3efc5132d None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:26:20.057970 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 11/21] 10.222.0.57 () {56 vars in 1322 bytes} [Fri Jul 1 16:26:19 2022] GET /identity/v3/role_assignments?user.id=9996f66c673f4673ab2342b66da9acc2&scope.domain.id=default&role.id=84ee739146934c1ab32c501ff8093644 => generated 233 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 01 16:26:22.430381 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b17d1fb-46ab-4b38-80b9-d4aa2166eaf8 None None] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:26:22.430883 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b17d1fb-46ab-4b38-80b9-d4aa2166eaf8 None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:26:22.430981 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b17d1fb-46ab-4b38-80b9-d4aa2166eaf8 None None] PATH_INFO: `/` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:26:22.432312 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 11/22] 10.222.0.57 () {54 vars in 926 bytes} [Fri Jul 1 16:26:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 16:26:22.438250 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915844f9-8569-4cb6-af09-1ef02090056b None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:26:22.439031 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915844f9-8569-4cb6-af09-1ef02090056b None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:26:22.439454 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915844f9-8569-4cb6-af09-1ef02090056b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:26:22.460108 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG passlib.handlers.bcrypt [None req-915844f9-8569-4cb6-af09-1ef02090056b None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80311) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jul 01 16:26:22.460807 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG passlib.handlers.bcrypt [None req-915844f9-8569-4cb6-af09-1ef02090056b None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80311) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jul 01 16:26:22.523461 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-915844f9-8569-4cb6-af09-1ef02090056b None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:26:22.525307 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-915844f9-8569-4cb6-af09-1ef02090056b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:26:22.629375 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 12/23] 10.222.0.57 () {58 vars in 1023 bytes} [Fri Jul 1 16:26:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 01 16:26:22.667395 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b17d1fb-46ab-4b38-80b9-d4aa2166eaf8 None None] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:26:22.679467 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bd8da1-72f3-4a85-a4f8-5437815663e1 None None] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:26:22.679949 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bd8da1-72f3-4a85-a4f8-5437815663e1 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:26:22.680444 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bd8da1-72f3-4a85-a4f8-5437815663e1 None None] PATH_INFO: `/` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:26:22.682369 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 13/24] 10.222.0.57 () {54 vars in 926 bytes} [Fri Jul 1 16:26:22 2022] GET /identity => generated 270 bytes in 14 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 16:26:22.688558 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e09b98-3d1a-4ec0-83cb-39a1004b07d0 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:26:22.689112 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e09b98-3d1a-4ec0-83cb-39a1004b07d0 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:26:22.689732 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e09b98-3d1a-4ec0-83cb-39a1004b07d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:26:22.716209 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-38e09b98-3d1a-4ec0-83cb-39a1004b07d0 None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:26:22.717939 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-38e09b98-3d1a-4ec0-83cb-39a1004b07d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:26:22.746977 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:26:22.849411 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 14/25] 10.222.0.57 () {58 vars in 1023 bytes} [Fri Jul 1 16:26:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 01 16:26:22.850804 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea46693-679f-4ea6-80a1-62639ffcc2be None None] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:26:22.851260 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea46693-679f-4ea6-80a1-62639ffcc2be None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:26:22.851690 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea46693-679f-4ea6-80a1-62639ffcc2be None None] PATH_INFO: `/` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:26:22.853105 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 15/26] 10.222.0.57 () {54 vars in 926 bytes} [Fri Jul 1 16:26:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 16:26:22.860105 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05230504-1789-4f1b-bdfb-eb95cd13b4db None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:26:22.860727 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05230504-1789-4f1b-bdfb-eb95cd13b4db None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:26:22.861211 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05230504-1789-4f1b-bdfb-eb95cd13b4db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:26:22.899572 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-05230504-1789-4f1b-bdfb-eb95cd13b4db None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:26:22.900774 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-05230504-1789-4f1b-bdfb-eb95cd13b4db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:26:22.965624 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b17d1fb-46ab-4b38-80b9-d4aa2166eaf8 None None] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:26:23.015480 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 16/27] 10.222.0.57 () {58 vars in 1023 bytes} [Fri Jul 1 16:26:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 01 16:26:23.017543 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05230504-1789-4f1b-bdfb-eb95cd13b4db None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:26:23.041291 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec950c10-d47f-460b-8299-37a8a5af26c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:26:23.042489 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec950c10-d47f-460b-8299-37a8a5af26c4 None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:26:23.043008 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec950c10-d47f-460b-8299-37a8a5af26c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:26:23.043515 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec950c10-d47f-460b-8299-37a8a5af26c4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:26:23.065508 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec950c10-d47f-460b-8299-37a8a5af26c4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:26:23.067351 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec950c10-d47f-460b-8299-37a8a5af26c4 None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:26:23.087114 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application [None req-ec950c10-d47f-460b-8299-37a8a5af26c4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 16:26:23.087114 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 16:26:23.087114 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 16:26:23.087114 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 16:26:23.087114 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 16:26:23.087114 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 16:26:23.087114 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 16:26:23.087114 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 16:26:23.087114 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 16:26:23.087114 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 16:26:23.087114 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 16:26:23.087114 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 16:26:23.087114 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 01 16:26:23.087114 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 01 16:26:23.087114 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 01 16:26:23.087974 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 01 16:26:23.087974 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 16:26:23.087974 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 16:26:23.087974 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 16:26:23.087974 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 16:26:23.087974 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 16:26:23.087974 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 16:26:23.087974 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 16:26:23.087974 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application with Lock( Jul 01 16:26:23.087974 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 16:26:23.087974 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self._enter() Jul 01 16:26:23.087974 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 16:26:23.087974 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 16:26:23.087974 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 16:26:23.087974 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.creator() Jul 01 16:26:23.087974 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 16:26:23.087974 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application created_value = creator( Jul 01 16:26:23.088802 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 01 16:26:23.088802 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 01 16:26:23.088802 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 16:26:23.088802 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application Jul 01 16:26:23.091380 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 12/28] 10.222.0.57 () {56 vars in 1108 bytes} [Fri Jul 1 16:26:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 425 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 16:26:23.092390 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec950c10-d47f-460b-8299-37a8a5af26c4 None admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:26:23.117352 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=100 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:26:23.129964 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:26:23.150129 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05230504-1789-4f1b-bdfb-eb95cd13b4db None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:26:23.196154 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e0813a-02b4-4b02-958f-7970037344a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:26:23.197481 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e0813a-02b4-4b02-958f-7970037344a7 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:26:23.197941 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e0813a-02b4-4b02-958f-7970037344a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:26:23.198655 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e0813a-02b4-4b02-958f-7970037344a7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:26:23.213509 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e0813a-02b4-4b02-958f-7970037344a7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 01 16:26:23.218059 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec950c10-d47f-460b-8299-37a8a5af26c4 None admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:26:23.223879 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application [None req-84e0813a-02b4-4b02-958f-7970037344a7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 01 16:26:23.223879 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 16:26:23.223879 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 16:26:23.223879 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 16:26:23.223879 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 16:26:23.223879 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 16:26:23.223879 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 16:26:23.223879 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 16:26:23.223879 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 16:26:23.223879 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 16:26:23.223879 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 16:26:23.223879 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 16:26:23.223879 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 01 16:26:23.223879 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 01 16:26:23.223879 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 01 16:26:23.223879 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 01 16:26:23.224946 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 01 16:26:23.224946 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 01 16:26:23.224946 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 01 16:26:23.224946 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 01 16:26:23.224946 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 16:26:23.224946 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 16:26:23.224946 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 16:26:23.224946 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 16:26:23.224946 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 16:26:23.224946 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 16:26:23.224946 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 16:26:23.224946 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application with Lock( Jul 01 16:26:23.224946 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 16:26:23.224946 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._enter() Jul 01 16:26:23.224946 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 16:26:23.224946 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 16:26:23.225870 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 16:26:23.225870 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.creator() Jul 01 16:26:23.225870 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 16:26:23.225870 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application created_value = creator( Jul 01 16:26:23.225870 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 01 16:26:23.225870 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 01 16:26:23.225870 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 01 16:26:23.225870 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 01 16:26:23.225870 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 01 16:26:23.225870 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 01 16:26:23.225870 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 01 16:26:23.225870 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Jul 01 16:26:23.226916 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 17/29] 10.222.0.57 () {56 vars in 1096 bytes} [Fri Jul 1 16:26:23 2022] GET /identity/v3/roles/admin => generated 83 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 16:26:23.228438 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3202a060-5cc0-4461-aaa5-8bce6d2a331b None None] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:26:23.228925 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3202a060-5cc0-4461-aaa5-8bce6d2a331b None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:26:23.229407 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3202a060-5cc0-4461-aaa5-8bce6d2a331b None None] PATH_INFO: `/` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:26:23.230922 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 18/30] 10.222.0.57 () {54 vars in 926 bytes} [Fri Jul 1 16:26:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 16:26:23.236340 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3202a060-5cc0-4461-aaa5-8bce6d2a331b None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:26:23.270071 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8d345a-bb04-4dbb-8c7e-3244c9ce00fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:26:23.270852 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8d345a-bb04-4dbb-8c7e-3244c9ce00fa None admin] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:26:23.271281 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8d345a-bb04-4dbb-8c7e-3244c9ce00fa None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:26:23.271819 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8d345a-bb04-4dbb-8c7e-3244c9ce00fa None admin] PATH_INFO: `/v3/roles` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:26:23.272804 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd8d345a-bb04-4dbb-8c7e-3244c9ce00fa None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:26:23.274525 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd8d345a-bb04-4dbb-8c7e-3244c9ce00fa None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:26:23.364786 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3202a060-5cc0-4461-aaa5-8bce6d2a331b None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:26:23.403907 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe432686-c448-4c49-99c7-4e9a8d6ab855 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:26:23.405117 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe432686-c448-4c49-99c7-4e9a8d6ab855 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:26:23.405621 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe432686-c448-4c49-99c7-4e9a8d6ab855 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:26:23.406113 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe432686-c448-4c49-99c7-4e9a8d6ab855 None admin] PATH_INFO: `/v3/domains` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:26:23.407216 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe432686-c448-4c49-99c7-4e9a8d6ab855 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:26:23.407741 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe432686-c448-4c49-99c7-4e9a8d6ab855 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:26:23.409466 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe432686-c448-4c49-99c7-4e9a8d6ab855 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:26:23.413050 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 13/31] 10.222.0.57 () {60 vars in 1131 bytes} [Fri Jul 1 16:26:23 2022] POST /identity/v3/roles => generated 219 bytes in 321 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 16:26:23.414109 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8d345a-bb04-4dbb-8c7e-3244c9ce00fa None admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:26:23.445778 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 19/32] 10.222.0.57 () {56 vars in 1109 bytes} [Fri Jul 1 16:26:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:26:23.450696 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cd4ec4-5258-48fe-8799-500dd119a7ed None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:26:23.450696 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cd4ec4-5258-48fe-8799-500dd119a7ed None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:26:23.450696 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cd4ec4-5258-48fe-8799-500dd119a7ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:26:23.473001 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:26:23.487510 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-00cd4ec4-5258-48fe-8799-500dd119a7ed None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:26:23.489414 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-00cd4ec4-5258-48fe-8799-500dd119a7ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:26:23.532124 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:26:23.584280 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8d345a-bb04-4dbb-8c7e-3244c9ce00fa None admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:26:23.644539 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 20/33] 10.222.0.57 () {58 vars in 1023 bytes} [Fri Jul 1 16:26:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 01 16:26:23.645477 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00cd4ec4-5258-48fe-8799-500dd119a7ed None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:26:23.648789 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d2931c-017c-42eb-a35a-ab7e1711abe9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:26:23.649648 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d2931c-017c-42eb-a35a-ab7e1711abe9 None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:26:23.650086 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d2931c-017c-42eb-a35a-ab7e1711abe9 None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:26:23.650581 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d2931c-017c-42eb-a35a-ab7e1711abe9 None admin] PATH_INFO: `/v3/roles` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:26:23.651849 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d2931c-017c-42eb-a35a-ab7e1711abe9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80312) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:26:23.652315 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d2931c-017c-42eb-a35a-ab7e1711abe9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:26:23.656695 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d2931c-017c-42eb-a35a-ab7e1711abe9 None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:26:23.672033 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 14/34] 10.222.0.57 () {56 vars in 1099 bytes} [Fri Jul 1 16:26:23 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:26:23.678584 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d2931c-017c-42eb-a35a-ab7e1711abe9 None admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:26:23.806283 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00cd4ec4-5258-48fe-8799-500dd119a7ed None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:26:23.823631 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d2931c-017c-42eb-a35a-ab7e1711abe9 None admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:26:23.860060 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f479634f-b4f9-46f0-98c1-9685082d040c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:26:23.861154 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f479634f-b4f9-46f0-98c1-9685082d040c None admin] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:26:23.861670 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f479634f-b4f9-46f0-98c1-9685082d040c None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:26:23.862148 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f479634f-b4f9-46f0-98c1-9685082d040c None admin] PATH_INFO: `/v3/projects` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:26:23.863393 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f479634f-b4f9-46f0-98c1-9685082d040c None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:26:23.865114 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f479634f-b4f9-46f0-98c1-9685082d040c None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:26:23.876205 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:26:23.987898 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7b2a22-e4e9-4824-85a3-43500e03b57f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:26:23.988860 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7b2a22-e4e9-4824-85a3-43500e03b57f None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:26:23.989258 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7b2a22-e4e9-4824-85a3-43500e03b57f None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:26:23.989630 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7b2a22-e4e9-4824-85a3-43500e03b57f None admin] PATH_INFO: `/v3/users/9996f66c673f4673ab2342b66da9acc2` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:26:23.990955 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b7b2a22-e4e9-4824-85a3-43500e03b57f None admin] RBAC: Authorizing `identity:get_user(user_id=9996f66c673f4673ab2342b66da9acc2)` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:26:23.992404 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b7b2a22-e4e9-4824-85a3-43500e03b57f None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:26:23.994091 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 15/35] 10.222.0.57 () {56 vars in 1177 bytes} [Fri Jul 1 16:26:23 2022] GET /identity/v3/users/9996f66c673f4673ab2342b66da9acc2 => generated 247 bytes in 317 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:26:23.994870 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7b2a22-e4e9-4824-85a3-43500e03b57f None admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:26:24.008562 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=100 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:26:24.046380 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 21/36] 10.222.0.57 () {60 vars in 1141 bytes} [Fri Jul 1 16:26:23 2022] POST /identity/v3/projects => generated 301 bytes in 396 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 16:26:24.046380 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854379ba-31e4-4d02-a273-c2ecf255e072 None None] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:26:24.046380 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854379ba-31e4-4d02-a273-c2ecf255e072 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:26:24.046380 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854379ba-31e4-4d02-a273-c2ecf255e072 None None] PATH_INFO: `/` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:26:24.046765 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 22/37] 10.222.0.57 () {54 vars in 926 bytes} [Fri Jul 1 16:26:24 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 16:26:24.054426 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854379ba-31e4-4d02-a273-c2ecf255e072 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:26:24.119519 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7b2a22-e4e9-4824-85a3-43500e03b57f None admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:26:24.167918 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4fed02-c0c4-4904-9fbb-bc5fdf57ab40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:26:24.168680 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4fed02-c0c4-4904-9fbb-bc5fdf57ab40 None admin] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:26:24.168968 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4fed02-c0c4-4904-9fbb-bc5fdf57ab40 None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:26:24.169253 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4fed02-c0c4-4904-9fbb-bc5fdf57ab40 None admin] PATH_INFO: `/v3/roles` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:26:24.170422 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf4fed02-c0c4-4904-9fbb-bc5fdf57ab40 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:26:24.171860 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf4fed02-c0c4-4904-9fbb-bc5fdf57ab40 None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:26:24.188501 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 16/38] 10.222.0.57 () {60 vars in 1131 bytes} [Fri Jul 1 16:26:23 2022] POST /identity/v3/roles => generated 225 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 16:26:24.190095 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71d3b65-1afc-4ea7-aed2-02433456ee94 None None] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:26:24.190501 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71d3b65-1afc-4ea7-aed2-02433456ee94 None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:26:24.190856 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71d3b65-1afc-4ea7-aed2-02433456ee94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:26:24.213671 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854379ba-31e4-4d02-a273-c2ecf255e072 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:26:24.215149 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.auth.core [None req-d71d3b65-1afc-4ea7-aed2-02433456ee94 None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80312) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:26:24.219127 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-d71d3b65-1afc-4ea7-aed2-02433456ee94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:26:24.271542 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c298b045-89f9-409e-a8cf-9999d7e904e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:26:24.272497 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c298b045-89f9-409e-a8cf-9999d7e904e3 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:26:24.272793 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c298b045-89f9-409e-a8cf-9999d7e904e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:26:24.273083 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c298b045-89f9-409e-a8cf-9999d7e904e3 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:26:24.274522 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c298b045-89f9-409e-a8cf-9999d7e904e3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:26:24.276153 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c298b045-89f9-409e-a8cf-9999d7e904e3 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:26:24.278160 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 23/39] 10.222.0.57 () {56 vars in 1108 bytes} [Fri Jul 1 16:26:24 2022] GET /identity/v3/domains/default => generated 199 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:26:24.279505 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3381a7a-1aba-41e7-a56b-c99fa0e7cc89 None None] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:26:24.279802 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3381a7a-1aba-41e7-a56b-c99fa0e7cc89 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:26:24.280087 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3381a7a-1aba-41e7-a56b-c99fa0e7cc89 None None] PATH_INFO: `/` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:26:24.281803 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 24/40] 10.222.0.57 () {54 vars in 926 bytes} [Fri Jul 1 16:26:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 16:26:24.287129 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3381a7a-1aba-41e7-a56b-c99fa0e7cc89 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:26:24.309694 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:26:24.321678 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 17/41] 10.222.0.57 () {58 vars in 1023 bytes} [Fri Jul 1 16:26:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 01 16:26:24.327916 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236f693d-2f73-45bb-9be0-3feb30d337e1 None None] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:26:24.328210 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236f693d-2f73-45bb-9be0-3feb30d337e1 None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:26:24.328488 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236f693d-2f73-45bb-9be0-3feb30d337e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:26:24.352084 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.auth.core [None req-236f693d-2f73-45bb-9be0-3feb30d337e1 None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80312) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:26:24.353443 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-236f693d-2f73-45bb-9be0-3feb30d337e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:26:24.372616 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:26:24.433009 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3381a7a-1aba-41e7-a56b-c99fa0e7cc89 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:26:24.438205 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 18/42] 10.222.0.57 () {58 vars in 1023 bytes} [Fri Jul 1 16:26:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 01 16:26:24.439478 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236f693d-2f73-45bb-9be0-3feb30d337e1 None None] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:26:24.470905 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7de518c-88e4-4e13-8383-e31384856bc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:26:24.471790 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7de518c-88e4-4e13-8383-e31384856bc7 None admin] REQUEST_METHOD: `PUT` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:26:24.472078 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7de518c-88e4-4e13-8383-e31384856bc7 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:26:24.472373 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7de518c-88e4-4e13-8383-e31384856bc7 None admin] PATH_INFO: `/v3/domains/default/users/9996f66c673f4673ab2342b66da9acc2/roles/84ee739146934c1ab32c501ff8093644` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:26:24.474783 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7de518c-88e4-4e13-8383-e31384856bc7 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=9996f66c673f4673ab2342b66da9acc2, role_id=84ee739146934c1ab32c501ff8093644)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:26:24.476276 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7de518c-88e4-4e13-8383-e31384856bc7 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:26:24.490694 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 25/43] 10.222.0.57 () {58 vars in 1361 bytes} [Fri Jul 1 16:26:24 2022] PUT /identity/v3/domains/default/users/9996f66c673f4673ab2342b66da9acc2/roles/84ee739146934c1ab32c501ff8093644 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 16:26:24.491718 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7de518c-88e4-4e13-8383-e31384856bc7 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:26:24.576467 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236f693d-2f73-45bb-9be0-3feb30d337e1 None None] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:26:24.612470 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=100 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:26:24.624691 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d980e26-1468-4cf5-af23-142a4375936d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:26:24.625707 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d980e26-1468-4cf5-af23-142a4375936d None admin] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:26:24.626178 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d980e26-1468-4cf5-af23-142a4375936d None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:26:24.626675 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d980e26-1468-4cf5-af23-142a4375936d None admin] PATH_INFO: `/v3/roles` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:26:24.627673 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d980e26-1468-4cf5-af23-142a4375936d None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:26:24.629402 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d980e26-1468-4cf5-af23-142a4375936d None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:26:24.646685 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 19/44] 10.222.0.57 () {60 vars in 1131 bytes} [Fri Jul 1 16:26:24 2022] POST /identity/v3/roles => generated 223 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 16:26:24.721318 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7de518c-88e4-4e13-8383-e31384856bc7 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:26:24.740819 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:26:24.763378 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9c346e-3cf6-4473-b088-eaa17d5889f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:26:24.764192 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9c346e-3cf6-4473-b088-eaa17d5889f5 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:26:24.764477 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9c346e-3cf6-4473-b088-eaa17d5889f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:26:24.764755 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9c346e-3cf6-4473-b088-eaa17d5889f5 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:26:24.766008 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f9c346e-3cf6-4473-b088-eaa17d5889f5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:26:24.767547 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f9c346e-3cf6-4473-b088-eaa17d5889f5 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:26:24.769237 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 26/45] 10.222.0.57 () {56 vars in 1108 bytes} [Fri Jul 1 16:26:24 2022] GET /identity/v3/domains/default => generated 199 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:26:24.775276 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d980e26-1468-4cf5-af23-142a4375936d None admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:26:24.893201 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d980e26-1468-4cf5-af23-142a4375936d None admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:26:24.925488 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac10ce98-b9df-4e6c-a775-71ae34b12365 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:26:24.926153 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac10ce98-b9df-4e6c-a775-71ae34b12365 None admin] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:26:24.926472 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac10ce98-b9df-4e6c-a775-71ae34b12365 None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:26:24.926754 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac10ce98-b9df-4e6c-a775-71ae34b12365 None admin] PATH_INFO: `/v3/projects` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:26:24.927476 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac10ce98-b9df-4e6c-a775-71ae34b12365 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:26:24.928909 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac10ce98-b9df-4e6c-a775-71ae34b12365 None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:26:24.957350 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 20/46] 10.222.0.57 () {60 vars in 1141 bytes} [Fri Jul 1 16:26:24 2022] POST /identity/v3/projects => generated 312 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 16:26:26.615228 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf6155c-dc0a-4d19-8955-03f41f60689d None None] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:26:26.615747 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf6155c-dc0a-4d19-8955-03f41f60689d None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:26:26.615856 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf6155c-dc0a-4d19-8955-03f41f60689d None None] PATH_INFO: `/` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:26:26.617153 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 27/47] 10.222.0.57 () {54 vars in 926 bytes} [Fri Jul 1 16:26:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 16:26:26.622942 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad918f5-c882-4357-8da2-2b33432aa09e None None] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:26:26.623384 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad918f5-c882-4357-8da2-2b33432aa09e None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:26:26.623518 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad918f5-c882-4357-8da2-2b33432aa09e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:26:26.640431 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.auth.core [None req-8ad918f5-c882-4357-8da2-2b33432aa09e None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80312) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:26:26.641600 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-8ad918f5-c882-4357-8da2-2b33432aa09e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:26:26.671247 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:26:26.711210 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 21/48] 10.222.0.57 () {58 vars in 1023 bytes} [Fri Jul 1 16:26:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 01 16:26:26.735424 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf6155c-dc0a-4d19-8955-03f41f60689d None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:26:26.841228 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf6155c-dc0a-4d19-8955-03f41f60689d None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:26:26.876546 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da567e19-d26c-4d0c-aea2-9e7034e91442 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:26:26.877265 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da567e19-d26c-4d0c-aea2-9e7034e91442 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:26:26.877552 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da567e19-d26c-4d0c-aea2-9e7034e91442 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:26:26.877831 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da567e19-d26c-4d0c-aea2-9e7034e91442 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:26:26.889498 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da567e19-d26c-4d0c-aea2-9e7034e91442 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 01 16:26:26.897254 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application [None req-da567e19-d26c-4d0c-aea2-9e7034e91442 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 01 16:26:26.897254 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 16:26:26.897254 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 16:26:26.897254 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 16:26:26.897254 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 16:26:26.897254 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 16:26:26.897254 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 16:26:26.897254 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 16:26:26.897254 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 16:26:26.897254 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 16:26:26.897254 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 16:26:26.897254 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 16:26:26.897254 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 01 16:26:26.897254 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 01 16:26:26.897254 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 01 16:26:26.897254 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 01 16:26:26.898152 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 01 16:26:26.898152 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 01 16:26:26.898152 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 01 16:26:26.898152 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 01 16:26:26.898152 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 16:26:26.898152 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 16:26:26.898152 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 16:26:26.898152 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 16:26:26.898152 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 16:26:26.898152 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 16:26:26.898152 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 16:26:26.898152 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application with Lock( Jul 01 16:26:26.898152 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 16:26:26.898152 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._enter() Jul 01 16:26:26.898152 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 16:26:26.898152 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 16:26:26.898959 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 16:26:26.898959 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.creator() Jul 01 16:26:26.898959 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 16:26:26.898959 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application created_value = creator( Jul 01 16:26:26.898959 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 01 16:26:26.898959 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 01 16:26:26.898959 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 01 16:26:26.898959 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 01 16:26:26.898959 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 01 16:26:26.898959 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 01 16:26:26.898959 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 01 16:26:26.898959 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Jul 01 16:26:26.898959 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 28/49] 10.222.0.57 () {56 vars in 1096 bytes} [Fri Jul 1 16:26:26 2022] GET /identity/v3/roles/admin => generated 83 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 16:26:26.906045 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad918f5-c882-4357-8da2-2b33432aa09e None None] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:26:27.004381 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad918f5-c882-4357-8da2-2b33432aa09e None None] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:26:27.037711 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91baf818-875c-417b-b26a-b0c9463c9ab5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:26:27.038406 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91baf818-875c-417b-b26a-b0c9463c9ab5 None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:26:27.038696 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91baf818-875c-417b-b26a-b0c9463c9ab5 None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:26:27.038973 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91baf818-875c-417b-b26a-b0c9463c9ab5 None admin] PATH_INFO: `/v3/roles` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:26:27.039728 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91baf818-875c-417b-b26a-b0c9463c9ab5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80312) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:26:27.040037 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91baf818-875c-417b-b26a-b0c9463c9ab5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:26:27.041426 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91baf818-875c-417b-b26a-b0c9463c9ab5 None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:26:27.050171 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 22/50] 10.222.0.57 () {56 vars in 1099 bytes} [Fri Jul 1 16:26:26 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:26:27.055760 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da567e19-d26c-4d0c-aea2-9e7034e91442 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:26:27.102533 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:26:27.122681 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe91399-f661-4b53-b256-ecb18911708c None None] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:26:27.122979 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe91399-f661-4b53-b256-ecb18911708c None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:26:27.123269 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe91399-f661-4b53-b256-ecb18911708c None None] PATH_INFO: `/` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:26:27.124658 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 23/51] 10.222.0.57 () {54 vars in 926 bytes} [Fri Jul 1 16:26:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 01 16:26:27.129627 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17dc44ef-03b1-4578-9593-4c8533638de4 None None] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:26:27.129864 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17dc44ef-03b1-4578-9593-4c8533638de4 None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:26:27.130147 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17dc44ef-03b1-4578-9593-4c8533638de4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:26:27.146808 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.auth.core [None req-17dc44ef-03b1-4578-9593-4c8533638de4 None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80312) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:26:27.148012 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-17dc44ef-03b1-4578-9593-4c8533638de4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:26:27.159920 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da567e19-d26c-4d0c-aea2-9e7034e91442 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:26:27.173287 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=100 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:26:27.196068 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d9cb78-c4d9-4e51-adc2-08850bb65815 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:26:27.196949 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d9cb78-c4d9-4e51-adc2-08850bb65815 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:26:27.197238 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d9cb78-c4d9-4e51-adc2-08850bb65815 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:26:27.197522 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d9cb78-c4d9-4e51-adc2-08850bb65815 None admin] PATH_INFO: `/v3/users/9996f66c673f4673ab2342b66da9acc2` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:26:27.198761 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d9cb78-c4d9-4e51-adc2-08850bb65815 None admin] RBAC: Authorizing `identity:get_user(user_id=9996f66c673f4673ab2342b66da9acc2)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:26:27.200393 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d9cb78-c4d9-4e51-adc2-08850bb65815 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:26:27.202050 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 29/52] 10.222.0.57 () {56 vars in 1177 bytes} [Fri Jul 1 16:26:27 2022] GET /identity/v3/users/9996f66c673f4673ab2342b66da9acc2 => generated 247 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 01 16:26:27.207681 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d9cb78-c4d9-4e51-adc2-08850bb65815 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:26:27.219267 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 24/53] 10.222.0.57 () {58 vars in 1023 bytes} [Fri Jul 1 16:26:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 01 16:26:27.244266 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17dc44ef-03b1-4578-9593-4c8533638de4 None None] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:26:27.310606 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d9cb78-c4d9-4e51-adc2-08850bb65815 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:26:27.324369 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:26:27.345650 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d2dc78-ac8e-40d6-a435-a13a496879cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:26:27.346204 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17dc44ef-03b1-4578-9593-4c8533638de4 None None] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:26:27.346467 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d2dc78-ac8e-40d6-a435-a13a496879cf None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:26:27.346756 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d2dc78-ac8e-40d6-a435-a13a496879cf None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:26:27.347034 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d2dc78-ac8e-40d6-a435-a13a496879cf None admin] PATH_INFO: `/v3/domains/default` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:26:27.348233 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d2dc78-ac8e-40d6-a435-a13a496879cf None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:26:27.349697 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d2dc78-ac8e-40d6-a435-a13a496879cf None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:26:27.351352 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 30/54] 10.222.0.57 () {56 vars in 1108 bytes} [Fri Jul 1 16:26:27 2022] GET /identity/v3/domains/default => generated 199 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 01 16:26:27.357776 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d2dc78-ac8e-40d6-a435-a13a496879cf None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:26:27.382444 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f4a429-d6f4-4404-84fb-028910daf86f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:26:27.383184 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f4a429-d6f4-4404-84fb-028910daf86f None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:26:27.383466 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f4a429-d6f4-4404-84fb-028910daf86f None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:26:27.383750 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f4a429-d6f4-4404-84fb-028910daf86f None admin] PATH_INFO: `/v3/domains/default` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:26:27.384928 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f4a429-d6f4-4404-84fb-028910daf86f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:26:27.386419 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f4a429-d6f4-4404-84fb-028910daf86f None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:26:27.388343 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 25/55] 10.222.0.57 () {56 vars in 1108 bytes} [Fri Jul 1 16:26:27 2022] GET /identity/v3/domains/default => generated 199 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:26:27.393529 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f4a429-d6f4-4404-84fb-028910daf86f None admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:26:27.421965 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=100 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:26:27.459850 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d2dc78-ac8e-40d6-a435-a13a496879cf None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:26:27.497558 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c056dd4-bcb0-437b-a329-89e34f2d740d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:26:27.498432 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c056dd4-bcb0-437b-a329-89e34f2d740d None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:26:27.498642 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c056dd4-bcb0-437b-a329-89e34f2d740d None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:26:27.498946 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c056dd4-bcb0-437b-a329-89e34f2d740d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:26:27.500025 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c056dd4-bcb0-437b-a329-89e34f2d740d None admin] RBAC: Adding query filter params (role.id=84ee739146934c1ab32c501ff8093644, scope.domain.id=default, user.id=9996f66c673f4673ab2342b66da9acc2) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:26:27.500434 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c056dd4-bcb0-437b-a329-89e34f2d740d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:26:27.502067 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f4a429-d6f4-4404-84fb-028910daf86f None admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:26:27.502560 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c056dd4-bcb0-437b-a329-89e34f2d740d None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:26:27.512264 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 31/56] 10.222.0.57 () {56 vars in 1322 bytes} [Fri Jul 1 16:26:27 2022] GET /identity/v3/role_assignments?user.id=9996f66c673f4673ab2342b66da9acc2&scope.domain.id=default&role.id=84ee739146934c1ab32c501ff8093644 => generated 530 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 01 16:26:27.539013 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6bd5e8f-2b77-48cf-ae8d-0ca3e4e20a47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:26:27.539409 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bd5e8f-2b77-48cf-ae8d-0ca3e4e20a47 None admin] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:26:27.539700 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bd5e8f-2b77-48cf-ae8d-0ca3e4e20a47 None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:26:27.539985 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bd5e8f-2b77-48cf-ae8d-0ca3e4e20a47 None admin] PATH_INFO: `/v3/projects` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:26:27.540739 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6bd5e8f-2b77-48cf-ae8d-0ca3e4e20a47 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:26:27.542260 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6bd5e8f-2b77-48cf-ae8d-0ca3e4e20a47 None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:26:27.572659 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 26/57] 10.222.0.57 () {60 vars in 1140 bytes} [Fri Jul 1 16:26:27 2022] POST /identity/v3/projects => generated 298 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 01 16:26:29.752465 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad4c772-b4bf-45f2-ae04-0dddbc9c4a81 None None] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:26:29.753015 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad4c772-b4bf-45f2-ae04-0dddbc9c4a81 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:26:29.753015 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad4c772-b4bf-45f2-ae04-0dddbc9c4a81 None None] PATH_INFO: `/` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:26:29.754435 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 32/58] 10.222.0.57 () {54 vars in 926 bytes} [Fri Jul 1 16:26:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 16:26:29.759657 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989776f1-b56c-4ce9-ba6e-ffbe9076623b None None] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:26:29.760079 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989776f1-b56c-4ce9-ba6e-ffbe9076623b None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:26:29.760220 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989776f1-b56c-4ce9-ba6e-ffbe9076623b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:26:29.776556 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.auth.core [None req-989776f1-b56c-4ce9-ba6e-ffbe9076623b None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80312) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:26:29.777772 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-989776f1-b56c-4ce9-ba6e-ffbe9076623b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:26:29.845339 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 27/59] 10.222.0.57 () {58 vars in 1023 bytes} [Fri Jul 1 16:26:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 01 16:26:29.870814 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad4c772-b4bf-45f2-ae04-0dddbc9c4a81 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:26:29.916567 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:26:29.972014 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad4c772-b4bf-45f2-ae04-0dddbc9c4a81 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:26:30.006395 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16c29fd-7210-4607-921f-b265ca074764 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:26:30.007129 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16c29fd-7210-4607-921f-b265ca074764 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:26:30.007494 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16c29fd-7210-4607-921f-b265ca074764 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:26:30.007778 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16c29fd-7210-4607-921f-b265ca074764 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:26:30.009172 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e16c29fd-7210-4607-921f-b265ca074764 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:26:30.010983 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e16c29fd-7210-4607-921f-b265ca074764 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:26:30.012990 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 33/60] 10.222.0.57 () {56 vars in 1108 bytes} [Fri Jul 1 16:26:29 2022] GET /identity/v3/domains/default => generated 199 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:26:30.018705 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989776f1-b56c-4ce9-ba6e-ffbe9076623b None None] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:26:30.031537 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:26:30.119864 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989776f1-b56c-4ce9-ba6e-ffbe9076623b None None] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:26:30.153116 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041a9739-1a1b-4da3-b76f-da06009f6282 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:26:30.153791 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041a9739-1a1b-4da3-b76f-da06009f6282 None admin] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:26:30.154083 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041a9739-1a1b-4da3-b76f-da06009f6282 None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:26:30.154392 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041a9739-1a1b-4da3-b76f-da06009f6282 None admin] PATH_INFO: `/v3/users` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:26:30.155120 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-041a9739-1a1b-4da3-b76f-da06009f6282 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:26:30.156572 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-041a9739-1a1b-4da3-b76f-da06009f6282 None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:26:30.181671 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 28/61] 10.222.0.57 () {60 vars in 1132 bytes} [Fri Jul 1 16:26:30 2022] POST /identity/v3/users => generated 275 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 16:26:33.565576 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8381c302-4a15-4659-86fd-a26502dea225 None None] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:26:33.565576 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8381c302-4a15-4659-86fd-a26502dea225 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:26:33.565576 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8381c302-4a15-4659-86fd-a26502dea225 None None] PATH_INFO: `/` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:26:33.565576 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 34/62] 10.222.0.57 () {54 vars in 926 bytes} [Fri Jul 1 16:26:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 16:26:33.570822 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990791fd-cfc5-4140-8f26-0a4d2d0c0a1a None None] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:26:33.571665 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990791fd-cfc5-4140-8f26-0a4d2d0c0a1a None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:26:33.572152 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990791fd-cfc5-4140-8f26-0a4d2d0c0a1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:26:33.592187 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.auth.core [None req-990791fd-cfc5-4140-8f26-0a4d2d0c0a1a None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80312) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:26:33.593728 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-990791fd-cfc5-4140-8f26-0a4d2d0c0a1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:26:33.630512 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe77861-a372-4337-b088-836d96e60814 None None] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:26:33.631129 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe77861-a372-4337-b088-836d96e60814 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:26:33.631612 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe77861-a372-4337-b088-836d96e60814 None None] PATH_INFO: `/` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:26:33.633199 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 35/63] 10.222.0.57 () {54 vars in 926 bytes} [Fri Jul 1 16:26:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 01 16:26:33.638609 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e96248-7931-4068-a52f-16f8827a5b0c None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:26:33.639190 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e96248-7931-4068-a52f-16f8827a5b0c None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:26:33.639839 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e96248-7931-4068-a52f-16f8827a5b0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:26:33.666665 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-d4e96248-7931-4068-a52f-16f8827a5b0c None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:26:33.669295 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-d4e96248-7931-4068-a52f-16f8827a5b0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:26:33.679152 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:26:33.689319 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 29/64] 10.222.0.57 () {58 vars in 1023 bytes} [Fri Jul 1 16:26:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 01 16:26:33.698372 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69582af4-6668-4959-8e32-ab11aebe45b0 None None] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:26:33.699181 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69582af4-6668-4959-8e32-ab11aebe45b0 None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:26:33.699619 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69582af4-6668-4959-8e32-ab11aebe45b0 None None] PATH_INFO: `/` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:26:33.701393 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 30/65] 10.222.0.57 () {54 vars in 926 bytes} [Fri Jul 1 16:26:33 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 16:26:33.707193 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4966bb-3725-496c-a98f-c8f3e095e0e8 None None] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:26:33.707701 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4966bb-3725-496c-a98f-c8f3e095e0e8 None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:26:33.708125 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4966bb-3725-496c-a98f-c8f3e095e0e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:26:33.736449 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.auth.core [None req-da4966bb-3725-496c-a98f-c8f3e095e0e8 None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80312) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:26:33.737778 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-da4966bb-3725-496c-a98f-c8f3e095e0e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:26:33.785405 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 36/66] 10.222.0.57 () {58 vars in 1023 bytes} [Fri Jul 1 16:26:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 01 16:26:33.787966 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e96248-7931-4068-a52f-16f8827a5b0c None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:26:33.855361 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 31/67] 10.222.0.57 () {58 vars in 1023 bytes} [Fri Jul 1 16:26:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 01 16:26:33.857700 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4966bb-3725-496c-a98f-c8f3e095e0e8 None None] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:26:33.866749 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:26:33.934913 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e96248-7931-4068-a52f-16f8827a5b0c None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:26:33.970292 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4966bb-3725-496c-a98f-c8f3e095e0e8 None None] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:26:33.971302 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff4f67f-823b-4177-b3a2-8d6c03b4e976 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:26:33.972116 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff4f67f-823b-4177-b3a2-8d6c03b4e976 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:26:33.972404 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff4f67f-823b-4177-b3a2-8d6c03b4e976 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:26:33.972687 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff4f67f-823b-4177-b3a2-8d6c03b4e976 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:26:33.983865 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff4f67f-823b-4177-b3a2-8d6c03b4e976 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 01 16:26:33.992024 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application [None req-dff4f67f-823b-4177-b3a2-8d6c03b4e976 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 01 16:26:33.992024 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 16:26:33.992024 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 16:26:33.992024 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 16:26:33.992024 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 16:26:33.992024 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 16:26:33.992024 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 16:26:33.992024 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 16:26:33.992024 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 16:26:33.992024 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 16:26:33.992024 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 16:26:33.992024 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 16:26:33.992024 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 01 16:26:33.992024 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 01 16:26:33.992024 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 01 16:26:33.992024 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 01 16:26:33.992882 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 01 16:26:33.992882 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 01 16:26:33.992882 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 01 16:26:33.992882 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 01 16:26:33.992882 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 16:26:33.992882 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 16:26:33.992882 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 16:26:33.992882 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 16:26:33.992882 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 16:26:33.992882 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 16:26:33.992882 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 16:26:33.992882 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application with Lock( Jul 01 16:26:33.992882 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 16:26:33.992882 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._enter() Jul 01 16:26:33.992882 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 16:26:33.992882 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 16:26:33.993643 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 16:26:33.993643 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.creator() Jul 01 16:26:33.993643 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 16:26:33.993643 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application created_value = creator( Jul 01 16:26:33.993643 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 01 16:26:33.993643 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 01 16:26:33.993643 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 01 16:26:33.993643 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 01 16:26:33.993643 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 01 16:26:33.993643 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 01 16:26:33.993643 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 01 16:26:33.993643 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Jul 01 16:26:33.993643 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 37/68] 10.222.0.57 () {56 vars in 1114 bytes} [Fri Jul 1 16:26:33 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 16:26:33.996911 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fd9727-12ec-4ec6-b41b-61da7f85176b None None] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:26:33.997656 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fd9727-12ec-4ec6-b41b-61da7f85176b None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:26:33.998122 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fd9727-12ec-4ec6-b41b-61da7f85176b None None] PATH_INFO: `/` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:26:34.000409 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 38/69] 10.222.0.57 () {54 vars in 926 bytes} [Fri Jul 1 16:26:33 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 16:26:34.003183 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13fd9727-12ec-4ec6-b41b-61da7f85176b None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:26:34.006269 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=100 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:26:34.016372 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e73ef6-e721-4c4a-b798-d03bb4b1bacc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:26:34.017187 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e73ef6-e721-4c4a-b798-d03bb4b1bacc None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:26:34.017477 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e73ef6-e721-4c4a-b798-d03bb4b1bacc None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:26:34.017759 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e73ef6-e721-4c4a-b798-d03bb4b1bacc None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:26:34.029948 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e73ef6-e721-4c4a-b798-d03bb4b1bacc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 01 16:26:34.038983 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application [None req-b3e73ef6-e721-4c4a-b798-d03bb4b1bacc None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 01 16:26:34.038983 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 16:26:34.038983 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 16:26:34.038983 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 16:26:34.038983 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 16:26:34.038983 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 16:26:34.038983 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 16:26:34.038983 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 16:26:34.038983 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 16:26:34.038983 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 16:26:34.038983 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 16:26:34.038983 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 16:26:34.038983 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 01 16:26:34.038983 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 01 16:26:34.038983 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 01 16:26:34.038983 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 01 16:26:34.039906 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 01 16:26:34.039906 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 01 16:26:34.039906 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 01 16:26:34.039906 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 01 16:26:34.039906 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 16:26:34.039906 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 16:26:34.039906 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 16:26:34.039906 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 16:26:34.039906 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 16:26:34.039906 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 16:26:34.039906 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 16:26:34.039906 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application with Lock( Jul 01 16:26:34.039906 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 16:26:34.039906 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self._enter() Jul 01 16:26:34.039906 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 16:26:34.039906 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 16:26:34.040682 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 16:26:34.040682 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.creator() Jul 01 16:26:34.040682 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 16:26:34.040682 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application created_value = creator( Jul 01 16:26:34.040682 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 01 16:26:34.040682 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 01 16:26:34.040682 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 01 16:26:34.040682 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 01 16:26:34.040682 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 01 16:26:34.040682 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 01 16:26:34.040682 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 01 16:26:34.040682 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application Jul 01 16:26:34.040682 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 32/70] 10.222.0.57 () {56 vars in 1096 bytes} [Fri Jul 1 16:26:33 2022] GET /identity/v3/roles/admin => generated 83 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 16:26:34.041326 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e73ef6-e721-4c4a-b798-d03bb4b1bacc None admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:26:34.119949 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13fd9727-12ec-4ec6-b41b-61da7f85176b None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:26:34.121763 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:26:34.149736 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e73ef6-e721-4c4a-b798-d03bb4b1bacc None admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:26:34.156828 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3874d417-6843-4a74-a9f0-863def433113 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:26:34.157633 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3874d417-6843-4a74-a9f0-863def433113 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:26:34.157921 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3874d417-6843-4a74-a9f0-863def433113 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:26:34.158205 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3874d417-6843-4a74-a9f0-863def433113 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:26:34.169642 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3874d417-6843-4a74-a9f0-863def433113 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 01 16:26:34.177331 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application [None req-3874d417-6843-4a74-a9f0-863def433113 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 01 16:26:34.177331 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 16:26:34.177331 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 16:26:34.177331 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 16:26:34.177331 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 16:26:34.177331 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 16:26:34.177331 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 16:26:34.177331 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 16:26:34.177331 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 16:26:34.177331 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 16:26:34.177331 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 16:26:34.177331 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 16:26:34.177331 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 01 16:26:34.177331 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 01 16:26:34.177331 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 01 16:26:34.177331 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 01 16:26:34.178286 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 01 16:26:34.178286 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 01 16:26:34.178286 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 01 16:26:34.178286 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 01 16:26:34.178286 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 16:26:34.178286 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 16:26:34.178286 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 16:26:34.178286 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 16:26:34.178286 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 16:26:34.178286 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 16:26:34.178286 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 16:26:34.178286 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application with Lock( Jul 01 16:26:34.178286 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 16:26:34.178286 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._enter() Jul 01 16:26:34.178286 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 16:26:34.178286 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 16:26:34.179088 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 16:26:34.179088 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.creator() Jul 01 16:26:34.179088 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 16:26:34.179088 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application created_value = creator( Jul 01 16:26:34.179088 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 01 16:26:34.179088 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 01 16:26:34.179088 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 01 16:26:34.179088 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 01 16:26:34.179088 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 01 16:26:34.179088 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 01 16:26:34.179088 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 01 16:26:34.179088 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Jul 01 16:26:34.179088 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 39/71] 10.222.0.57 () {56 vars in 1099 bytes} [Fri Jul 1 16:26:34 2022] GET /identity/v3/roles/member => generated 84 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 16:26:34.179949 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac11ec78-c3fd-4d70-98c8-b470ab2192e7 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:26:34.180195 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac11ec78-c3fd-4d70-98c8-b470ab2192e7 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:26:34.180457 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac11ec78-c3fd-4d70-98c8-b470ab2192e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:26:34.189299 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a82cad-001c-45b6-85ec-9d53e85a2836 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:26:34.190060 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a82cad-001c-45b6-85ec-9d53e85a2836 None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:26:34.190457 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a82cad-001c-45b6-85ec-9d53e85a2836 None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:26:34.190747 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a82cad-001c-45b6-85ec-9d53e85a2836 None admin] PATH_INFO: `/v3/roles` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:26:34.191521 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a82cad-001c-45b6-85ec-9d53e85a2836 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80312) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:26:34.191840 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a82cad-001c-45b6-85ec-9d53e85a2836 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:26:34.193255 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a82cad-001c-45b6-85ec-9d53e85a2836 None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:26:34.197629 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-ac11ec78-c3fd-4d70-98c8-b470ab2192e7 None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:26:34.198873 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-ac11ec78-c3fd-4d70-98c8-b470ab2192e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:26:34.202274 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 33/72] 10.222.0.57 () {56 vars in 1111 bytes} [Fri Jul 1 16:26:34 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:26:34.203577 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6baa01c8-c339-45d7-a645-2e96071f7e21 None None] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:26:34.203875 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6baa01c8-c339-45d7-a645-2e96071f7e21 None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:26:34.204164 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6baa01c8-c339-45d7-a645-2e96071f7e21 None None] PATH_INFO: `/` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:26:34.205412 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 34/73] 10.222.0.57 () {54 vars in 926 bytes} [Fri Jul 1 16:26:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 16:26:34.206380 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6baa01c8-c339-45d7-a645-2e96071f7e21 None None] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:26:34.256386 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=100 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:26:34.270465 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 40/74] 10.222.0.57 () {58 vars in 1023 bytes} [Fri Jul 1 16:26:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 01 16:26:34.279306 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac11ec78-c3fd-4d70-98c8-b470ab2192e7 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:26:34.313833 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6baa01c8-c339-45d7-a645-2e96071f7e21 None None] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:26:34.350169 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc540cd9-6369-447c-8e89-b7f1a4a14d82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:26:34.350902 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc540cd9-6369-447c-8e89-b7f1a4a14d82 None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:26:34.351190 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc540cd9-6369-447c-8e89-b7f1a4a14d82 None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:26:34.351468 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc540cd9-6369-447c-8e89-b7f1a4a14d82 None admin] PATH_INFO: `/v3/roles` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:26:34.352224 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc540cd9-6369-447c-8e89-b7f1a4a14d82 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80312) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:26:34.352535 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc540cd9-6369-447c-8e89-b7f1a4a14d82 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:26:34.354143 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc540cd9-6369-447c-8e89-b7f1a4a14d82 None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:26:34.364924 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 35/75] 10.222.0.57 () {56 vars in 1099 bytes} [Fri Jul 1 16:26:34 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:26:34.365568 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc540cd9-6369-447c-8e89-b7f1a4a14d82 None admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:26:34.385478 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:26:34.394942 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac11ec78-c3fd-4d70-98c8-b470ab2192e7 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:26:34.435286 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008920bd-6a76-43d7-82e9-15292ef64291 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:26:34.435961 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008920bd-6a76-43d7-82e9-15292ef64291 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:26:34.436253 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008920bd-6a76-43d7-82e9-15292ef64291 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:26:34.436537 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008920bd-6a76-43d7-82e9-15292ef64291 None admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:26:34.437304 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-008920bd-6a76-43d7-82e9-15292ef64291 None admin] RBAC: Adding query filter params (name=member) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:26:34.437619 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-008920bd-6a76-43d7-82e9-15292ef64291 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:26:34.439048 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-008920bd-6a76-43d7-82e9-15292ef64291 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:26:34.447257 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 41/76] 10.222.0.57 () {56 vars in 1101 bytes} [Fri Jul 1 16:26:34 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:26:34.448466 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771204cd-65be-45bf-8117-596a94c3226a None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:26:34.448765 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771204cd-65be-45bf-8117-596a94c3226a None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:26:34.449077 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771204cd-65be-45bf-8117-596a94c3226a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:26:34.465812 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-771204cd-65be-45bf-8117-596a94c3226a None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:26:34.467025 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-771204cd-65be-45bf-8117-596a94c3226a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:26:34.473279 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc540cd9-6369-447c-8e89-b7f1a4a14d82 None admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:26:34.484127 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=100 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:26:34.508082 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2096ed80-b9ca-42bb-baa2-4e6762d4a444 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:26:34.508980 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2096ed80-b9ca-42bb-baa2-4e6762d4a444 None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:26:34.509273 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2096ed80-b9ca-42bb-baa2-4e6762d4a444 None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:26:34.509558 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2096ed80-b9ca-42bb-baa2-4e6762d4a444 None admin] PATH_INFO: `/v3/users/dcca9b98066044dca1d7d028b9e83c21` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:26:34.524253 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2096ed80-b9ca-42bb-baa2-4e6762d4a444 None admin] RBAC: Authorizing `identity:get_user(user_id=dcca9b98066044dca1d7d028b9e83c21)` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:26:34.525739 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2096ed80-b9ca-42bb-baa2-4e6762d4a444 None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:26:34.528298 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 36/77] 10.222.0.57 () {56 vars in 1177 bytes} [Fri Jul 1 16:26:34 2022] GET /identity/v3/users/dcca9b98066044dca1d7d028b9e83c21 => generated 275 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:26:34.529186 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2096ed80-b9ca-42bb-baa2-4e6762d4a444 None admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:26:34.536216 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 42/78] 10.222.0.57 () {58 vars in 1023 bytes} [Fri Jul 1 16:26:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 01 16:26:34.537068 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771204cd-65be-45bf-8117-596a94c3226a None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:26:34.623333 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:26:34.630821 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2096ed80-b9ca-42bb-baa2-4e6762d4a444 None admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:26:34.640925 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771204cd-65be-45bf-8117-596a94c3226a None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:26:34.665829 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3f9692-5a92-4322-b48a-43600b57ffb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:26:34.666667 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3f9692-5a92-4322-b48a-43600b57ffb5 None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:26:34.666964 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3f9692-5a92-4322-b48a-43600b57ffb5 None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:26:34.667249 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3f9692-5a92-4322-b48a-43600b57ffb5 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:26:34.669687 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b3f9692-5a92-4322-b48a-43600b57ffb5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:26:34.671812 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b3f9692-5a92-4322-b48a-43600b57ffb5 None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:26:34.673529 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 37/79] 10.222.0.57 () {56 vars in 1108 bytes} [Fri Jul 1 16:26:34 2022] GET /identity/v3/domains/default => generated 199 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:26:34.678043 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3f9692-5a92-4322-b48a-43600b57ffb5 None admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:26:34.678149 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd707fba-66d8-4d20-814c-a299ca03777b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:26:34.678149 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd707fba-66d8-4d20-814c-a299ca03777b None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:26:34.678149 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd707fba-66d8-4d20-814c-a299ca03777b None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:26:34.678149 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd707fba-66d8-4d20-814c-a299ca03777b None admin] PATH_INFO: `/v3/users/9996f66c673f4673ab2342b66da9acc2` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:26:34.679410 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd707fba-66d8-4d20-814c-a299ca03777b None admin] RBAC: Authorizing `identity:get_user(user_id=9996f66c673f4673ab2342b66da9acc2)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:26:34.680849 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd707fba-66d8-4d20-814c-a299ca03777b None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:26:34.682502 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 43/80] 10.222.0.57 () {56 vars in 1177 bytes} [Fri Jul 1 16:26:34 2022] GET /identity/v3/users/9996f66c673f4673ab2342b66da9acc2 => generated 247 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:26:34.683861 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd707fba-66d8-4d20-814c-a299ca03777b None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:26:34.711055 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=100 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:26:34.777765 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3f9692-5a92-4322-b48a-43600b57ffb5 None admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:26:34.797334 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd707fba-66d8-4d20-814c-a299ca03777b None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:26:34.811469 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de28fd1-77ef-4bde-b7ae-7570bedcbd47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:26:34.812348 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de28fd1-77ef-4bde-b7ae-7570bedcbd47 None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:26:34.812634 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de28fd1-77ef-4bde-b7ae-7570bedcbd47 None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:26:34.812914 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de28fd1-77ef-4bde-b7ae-7570bedcbd47 None admin] PATH_INFO: `/v3/users/dcca9b98066044dca1d7d028b9e83c21` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:26:34.814100 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de28fd1-77ef-4bde-b7ae-7570bedcbd47 None admin] RBAC: Authorizing `identity:get_user(user_id=dcca9b98066044dca1d7d028b9e83c21)` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:26:34.815602 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de28fd1-77ef-4bde-b7ae-7570bedcbd47 None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:26:34.817228 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 38/81] 10.222.0.57 () {56 vars in 1177 bytes} [Fri Jul 1 16:26:34 2022] GET /identity/v3/users/dcca9b98066044dca1d7d028b9e83c21 => generated 275 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:26:34.817991 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de28fd1-77ef-4bde-b7ae-7570bedcbd47 None admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:26:34.833251 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0550ceeb-2e85-48df-87d6-8b66738f70e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:26:34.833831 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0550ceeb-2e85-48df-87d6-8b66738f70e7 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:26:34.834122 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0550ceeb-2e85-48df-87d6-8b66738f70e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:26:34.834443 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0550ceeb-2e85-48df-87d6-8b66738f70e7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:26:34.857683 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:26:34.907167 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0550ceeb-2e85-48df-87d6-8b66738f70e7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:26:34.908780 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0550ceeb-2e85-48df-87d6-8b66738f70e7 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:26:34.927559 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de28fd1-77ef-4bde-b7ae-7570bedcbd47 None admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:26:34.929813 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=100 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:26:34.961896 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0aff0ad-f386-4ed5-81d2-f7e28f08d632 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:26:34.962742 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0aff0ad-f386-4ed5-81d2-f7e28f08d632 None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:26:34.963265 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0aff0ad-f386-4ed5-81d2-f7e28f08d632 None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:26:34.963372 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0aff0ad-f386-4ed5-81d2-f7e28f08d632 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:26:34.974305 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0aff0ad-f386-4ed5-81d2-f7e28f08d632 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 01 16:26:34.982179 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application [None req-e0aff0ad-f386-4ed5-81d2-f7e28f08d632 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 01 16:26:34.982179 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 16:26:34.982179 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 16:26:34.982179 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 16:26:34.982179 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 16:26:34.982179 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 16:26:34.982179 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 16:26:34.982179 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 16:26:34.982179 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 16:26:34.982179 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 16:26:34.982179 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 16:26:34.982179 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 16:26:34.982179 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 01 16:26:34.982179 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 01 16:26:34.982179 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 01 16:26:34.982179 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 01 16:26:34.983079 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 01 16:26:34.983079 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 01 16:26:34.983079 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 01 16:26:34.983079 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 01 16:26:34.983079 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 16:26:34.983079 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 16:26:34.983079 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 16:26:34.983079 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 16:26:34.983079 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 16:26:34.983079 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 16:26:34.983079 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 16:26:34.983079 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application with Lock( Jul 01 16:26:34.983079 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 16:26:34.983079 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self._enter() Jul 01 16:26:34.983079 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 16:26:34.983079 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 16:26:34.983840 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 16:26:34.983840 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.creator() Jul 01 16:26:34.983840 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 16:26:34.983840 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application created_value = creator( Jul 01 16:26:34.983840 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 01 16:26:34.983840 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 01 16:26:34.983840 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 01 16:26:34.983840 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 01 16:26:34.983840 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 01 16:26:34.983840 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 01 16:26:34.983840 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 01 16:26:34.983840 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application Jul 01 16:26:34.983840 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 39/82] 10.222.0.57 () {56 vars in 1099 bytes} [Fri Jul 1 16:26:34 2022] GET /identity/v3/roles/member => generated 84 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 16:26:34.984456 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0aff0ad-f386-4ed5-81d2-f7e28f08d632 None admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:26:35.015483 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 44/83] 10.222.0.57 () {58 vars in 1282 bytes} [Fri Jul 1 16:26:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 01 16:26:35.016332 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0550ceeb-2e85-48df-87d6-8b66738f70e7 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:26:35.067178 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:26:35.085672 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0aff0ad-f386-4ed5-81d2-f7e28f08d632 None admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:26:35.114829 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0550ceeb-2e85-48df-87d6-8b66738f70e7 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:26:35.120106 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34797e0-8540-4faa-a671-6102d57c14bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:26:35.120809 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34797e0-8540-4faa-a671-6102d57c14bf None admin] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:26:35.121145 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34797e0-8540-4faa-a671-6102d57c14bf None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:26:35.121480 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34797e0-8540-4faa-a671-6102d57c14bf None admin] PATH_INFO: `/v3/users` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:26:35.122201 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a34797e0-8540-4faa-a671-6102d57c14bf None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:26:35.123767 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a34797e0-8540-4faa-a671-6102d57c14bf None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:26:35.144311 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 40/84] 10.222.0.57 () {60 vars in 1132 bytes} [Fri Jul 1 16:26:34 2022] POST /identity/v3/users => generated 289 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 16:26:35.145195 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34797e0-8540-4faa-a671-6102d57c14bf None admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:26:35.150396 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f91c3c1-3cf4-4686-a800-56953b315bcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:26:35.151129 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f91c3c1-3cf4-4686-a800-56953b315bcf None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:26:35.151421 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f91c3c1-3cf4-4686-a800-56953b315bcf None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:26:35.151699 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f91c3c1-3cf4-4686-a800-56953b315bcf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:26:35.172416 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:26:35.232508 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f91c3c1-3cf4-4686-a800-56953b315bcf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:26:35.234018 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f91c3c1-3cf4-4686-a800-56953b315bcf None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:26:35.269891 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34797e0-8540-4faa-a671-6102d57c14bf None admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:26:35.292100 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:26:35.311203 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef4d2cc-2fa6-4d1c-8278-124cdd32648a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:26:35.311864 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef4d2cc-2fa6-4d1c-8278-124cdd32648a None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:26:35.312175 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef4d2cc-2fa6-4d1c-8278-124cdd32648a None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:26:35.312474 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef4d2cc-2fa6-4d1c-8278-124cdd32648a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:26:35.361906 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 45/85] 10.222.0.57 () {58 vars in 1282 bytes} [Fri Jul 1 16:26:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 01 16:26:35.361906 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f91c3c1-3cf4-4686-a800-56953b315bcf None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:26:35.403150 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef4d2cc-2fa6-4d1c-8278-124cdd32648a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:26:35.404773 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef4d2cc-2fa6-4d1c-8278-124cdd32648a None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:26:35.432057 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=100 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:26:35.487162 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f91c3c1-3cf4-4686-a800-56953b315bcf None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:26:35.527903 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 41/86] 10.222.0.57 () {58 vars in 1282 bytes} [Fri Jul 1 16:26:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 383 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 01 16:26:35.533332 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef4d2cc-2fa6-4d1c-8278-124cdd32648a None admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:26:35.538177 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c835c57-f58d-4194-a073-982a5a2f0ae1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:26:35.538883 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c835c57-f58d-4194-a073-982a5a2f0ae1 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:26:35.539171 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c835c57-f58d-4194-a073-982a5a2f0ae1 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:26:35.539452 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c835c57-f58d-4194-a073-982a5a2f0ae1 None admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:26:35.540214 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c835c57-f58d-4194-a073-982a5a2f0ae1 None admin] RBAC: Adding query filter params (name=member) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:26:35.540524 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c835c57-f58d-4194-a073-982a5a2f0ae1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:26:35.541924 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c835c57-f58d-4194-a073-982a5a2f0ae1 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:26:35.554986 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 46/87] 10.222.0.57 () {56 vars in 1101 bytes} [Fri Jul 1 16:26:35 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:26:35.555648 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c835c57-f58d-4194-a073-982a5a2f0ae1 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:26:35.579166 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:26:35.658384 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef4d2cc-2fa6-4d1c-8278-124cdd32648a None admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:26:35.675902 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c835c57-f58d-4194-a073-982a5a2f0ae1 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:26:35.685566 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=100 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:26:35.703203 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552e1552-4d7c-4b2d-8dfa-e8d4231ad544 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:26:35.704305 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552e1552-4d7c-4b2d-8dfa-e8d4231ad544 None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:26:35.704595 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552e1552-4d7c-4b2d-8dfa-e8d4231ad544 None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:26:35.704880 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552e1552-4d7c-4b2d-8dfa-e8d4231ad544 None admin] PATH_INFO: `/v3/projects/5bef204363a04d5d8ceab527b8803f6e` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:26:35.706336 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-552e1552-4d7c-4b2d-8dfa-e8d4231ad544 None admin] RBAC: Authorizing `identity:get_project(project_id=5bef204363a04d5d8ceab527b8803f6e)` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:26:35.708104 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-552e1552-4d7c-4b2d-8dfa-e8d4231ad544 None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:26:35.710610 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 42/88] 10.222.0.57 () {56 vars in 1186 bytes} [Fri Jul 1 16:26:35 2022] GET /identity/v3/projects/5bef204363a04d5d8ceab527b8803f6e => generated 298 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:26:35.711413 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552e1552-4d7c-4b2d-8dfa-e8d4231ad544 None admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:26:35.716815 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f0b2f6-cbe4-4101-99f2-f115075899ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:26:35.717989 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f0b2f6-cbe4-4101-99f2-f115075899ce None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:26:35.718284 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f0b2f6-cbe4-4101-99f2-f115075899ce None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:26:35.718778 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f0b2f6-cbe4-4101-99f2-f115075899ce None admin] PATH_INFO: `/v3/projects/5bef204363a04d5d8ceab527b8803f6e` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:26:35.720943 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f0b2f6-cbe4-4101-99f2-f115075899ce None admin] RBAC: Authorizing `identity:get_project(project_id=5bef204363a04d5d8ceab527b8803f6e)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:26:35.722442 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f0b2f6-cbe4-4101-99f2-f115075899ce None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:26:35.725127 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 47/89] 10.222.0.57 () {56 vars in 1186 bytes} [Fri Jul 1 16:26:35 2022] GET /identity/v3/projects/5bef204363a04d5d8ceab527b8803f6e => generated 298 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:26:35.725960 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f0b2f6-cbe4-4101-99f2-f115075899ce None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:26:35.834730 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:26:35.858958 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552e1552-4d7c-4b2d-8dfa-e8d4231ad544 None admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:26:35.863962 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f0b2f6-cbe4-4101-99f2-f115075899ce None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:26:35.894869 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72f3b14-a876-49fe-84a7-8774c6d49b91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:26:35.895762 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72f3b14-a876-49fe-84a7-8774c6d49b91 None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:26:35.896045 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72f3b14-a876-49fe-84a7-8774c6d49b91 None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:26:35.896330 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72f3b14-a876-49fe-84a7-8774c6d49b91 None admin] PATH_INFO: `/v3/projects/5bef204363a04d5d8ceab527b8803f6e` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:26:35.897581 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a72f3b14-a876-49fe-84a7-8774c6d49b91 None admin] RBAC: Authorizing `identity:get_project(project_id=5bef204363a04d5d8ceab527b8803f6e)` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:26:35.899090 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a72f3b14-a876-49fe-84a7-8774c6d49b91 None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:26:35.900820 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 43/90] 10.222.0.57 () {56 vars in 1186 bytes} [Fri Jul 1 16:26:35 2022] GET /identity/v3/projects/5bef204363a04d5d8ceab527b8803f6e => generated 298 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:26:35.901599 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72f3b14-a876-49fe-84a7-8774c6d49b91 None admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:26:35.910965 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44a7034-950c-4e40-b241-3aa6833e706f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:26:35.912632 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44a7034-950c-4e40-b241-3aa6833e706f None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:26:35.913115 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44a7034-950c-4e40-b241-3aa6833e706f None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:26:35.914081 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44a7034-950c-4e40-b241-3aa6833e706f None admin] PATH_INFO: `/v3/users/dcca9b98066044dca1d7d028b9e83c21` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:26:35.916263 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44a7034-950c-4e40-b241-3aa6833e706f None admin] RBAC: Authorizing `identity:get_user(user_id=dcca9b98066044dca1d7d028b9e83c21)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:26:35.919444 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44a7034-950c-4e40-b241-3aa6833e706f None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:26:35.922169 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 48/91] 10.222.0.57 () {56 vars in 1177 bytes} [Fri Jul 1 16:26:35 2022] GET /identity/v3/users/dcca9b98066044dca1d7d028b9e83c21 => generated 275 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:26:35.923295 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44a7034-950c-4e40-b241-3aa6833e706f None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:26:35.961790 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=100 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:26:36.008586 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72f3b14-a876-49fe-84a7-8774c6d49b91 None admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:26:36.030960 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44a7034-950c-4e40-b241-3aa6833e706f None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:26:36.045591 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859b3b88-1fca-4f84-8504-4d7bf720e7a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:26:36.046508 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859b3b88-1fca-4f84-8504-4d7bf720e7a1 None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:26:36.046639 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859b3b88-1fca-4f84-8504-4d7bf720e7a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:26:36.046937 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859b3b88-1fca-4f84-8504-4d7bf720e7a1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:26:36.047842 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-859b3b88-1fca-4f84-8504-4d7bf720e7a1 None admin] RBAC: Adding query filter params (role.id=a8e5c73da81d4f0eb7b92c7058e500fe, scope.project.id=5bef204363a04d5d8ceab527b8803f6e, user.id=dcca9b98066044dca1d7d028b9e83c21) {{(pid=80312) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:26:36.048176 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-859b3b88-1fca-4f84-8504-4d7bf720e7a1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:26:36.049735 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-859b3b88-1fca-4f84-8504-4d7bf720e7a1 None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:26:36.057165 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 44/92] 10.222.0.57 () {56 vars in 1374 bytes} [Fri Jul 1 16:26:35 2022] GET /identity/v3/role_assignments?user.id=dcca9b98066044dca1d7d028b9e83c21&scope.project.id=5bef204363a04d5d8ceab527b8803f6e&role.id=a8e5c73da81d4f0eb7b92c7058e500fe => generated 259 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:26:36.062626 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859b3b88-1fca-4f84-8504-4d7bf720e7a1 None admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:26:36.091151 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3163136-aab4-4f25-bb75-fbc0ffe46834 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:26:36.091875 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3163136-aab4-4f25-bb75-fbc0ffe46834 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:26:36.092178 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3163136-aab4-4f25-bb75-fbc0ffe46834 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:26:36.092478 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3163136-aab4-4f25-bb75-fbc0ffe46834 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:26:36.093413 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3163136-aab4-4f25-bb75-fbc0ffe46834 None admin] RBAC: Adding query filter params (role.id=84ee739146934c1ab32c501ff8093644, scope.project.id=5bef204363a04d5d8ceab527b8803f6e, user.id=9996f66c673f4673ab2342b66da9acc2) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:26:36.093759 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3163136-aab4-4f25-bb75-fbc0ffe46834 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:26:36.095402 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3163136-aab4-4f25-bb75-fbc0ffe46834 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:26:36.113829 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 49/93] 10.222.0.57 () {56 vars in 1374 bytes} [Fri Jul 1 16:26:35 2022] GET /identity/v3/role_assignments?user.id=9996f66c673f4673ab2342b66da9acc2&scope.project.id=5bef204363a04d5d8ceab527b8803f6e&role.id=84ee739146934c1ab32c501ff8093644 => generated 259 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:26:36.114810 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3163136-aab4-4f25-bb75-fbc0ffe46834 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:26:36.164199 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:26:36.223416 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859b3b88-1fca-4f84-8504-4d7bf720e7a1 None admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:26:36.250770 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=100 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:26:36.276427 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6864ad-2552-48e5-9346-50e7a82cbc74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:26:36.277270 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6864ad-2552-48e5-9346-50e7a82cbc74 None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:26:36.277702 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6864ad-2552-48e5-9346-50e7a82cbc74 None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:26:36.278000 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6864ad-2552-48e5-9346-50e7a82cbc74 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:26:36.278981 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6864ad-2552-48e5-9346-50e7a82cbc74 None admin] RBAC: Adding query filter params (role.id=983be40bdab04c478eaa90c6b6a46b14, scope.project.id=5bef204363a04d5d8ceab527b8803f6e, user.id=dcca9b98066044dca1d7d028b9e83c21) {{(pid=80312) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:26:36.279306 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6864ad-2552-48e5-9346-50e7a82cbc74 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:26:36.280922 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6864ad-2552-48e5-9346-50e7a82cbc74 None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:26:36.296850 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 45/94] 10.222.0.57 () {56 vars in 1374 bytes} [Fri Jul 1 16:26:36 2022] GET /identity/v3/role_assignments?user.id=dcca9b98066044dca1d7d028b9e83c21&scope.project.id=5bef204363a04d5d8ceab527b8803f6e&role.id=983be40bdab04c478eaa90c6b6a46b14 => generated 259 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:26:36.352333 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3163136-aab4-4f25-bb75-fbc0ffe46834 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:26:36.391759 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a6141d-4a41-4c5b-951f-affa05a820b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:26:36.392471 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a6141d-4a41-4c5b-951f-affa05a820b2 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:26:36.392846 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a6141d-4a41-4c5b-951f-affa05a820b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:26:36.393155 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a6141d-4a41-4c5b-951f-affa05a820b2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:26:36.487693 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a6141d-4a41-4c5b-951f-affa05a820b2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:26:36.489196 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a6141d-4a41-4c5b-951f-affa05a820b2 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:26:36.508719 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:26:36.627314 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 50/95] 10.222.0.57 () {58 vars in 1282 bytes} [Fri Jul 1 16:26:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 513 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 01 16:26:36.633224 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6864ad-2552-48e5-9346-50e7a82cbc74 None admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:26:36.772463 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6864ad-2552-48e5-9346-50e7a82cbc74 None admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:26:36.821682 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ed39c7-e36e-4c14-bc94-c6a1e4f12f17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:26:36.822585 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ed39c7-e36e-4c14-bc94-c6a1e4f12f17 None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:26:36.822873 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ed39c7-e36e-4c14-bc94-c6a1e4f12f17 None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:26:36.823154 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ed39c7-e36e-4c14-bc94-c6a1e4f12f17 None admin] PATH_INFO: `/v3/projects/bf1bfeff7565472da796cac2f99ad6bb` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:26:36.825157 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ed39c7-e36e-4c14-bc94-c6a1e4f12f17 None admin] RBAC: Authorizing `identity:get_project(project_id=bf1bfeff7565472da796cac2f99ad6bb)` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:26:36.826783 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ed39c7-e36e-4c14-bc94-c6a1e4f12f17 None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:26:36.828822 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 46/96] 10.222.0.57 () {56 vars in 1186 bytes} [Fri Jul 1 16:26:36 2022] GET /identity/v3/projects/bf1bfeff7565472da796cac2f99ad6bb => generated 312 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:26:36.834796 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a6141d-4a41-4c5b-951f-affa05a820b2 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:26:37.051638 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a6141d-4a41-4c5b-951f-affa05a820b2 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:26:37.075492 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:26:37.102891 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a10722-c547-4de6-a9e7-422fac57a83f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:26:37.103560 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a10722-c547-4de6-a9e7-422fac57a83f None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:26:37.103853 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a10722-c547-4de6-a9e7-422fac57a83f None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:26:37.104136 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a10722-c547-4de6-a9e7-422fac57a83f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:26:37.104981 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a10722-c547-4de6-a9e7-422fac57a83f None admin] RBAC: Adding query filter params (role.id=983be40bdab04c478eaa90c6b6a46b14, scope.project.id=bf1bfeff7565472da796cac2f99ad6bb, user.id=dcca9b98066044dca1d7d028b9e83c21) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:26:37.105403 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a10722-c547-4de6-a9e7-422fac57a83f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:26:37.106991 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a10722-c547-4de6-a9e7-422fac57a83f None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:26:37.115567 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 51/97] 10.222.0.57 () {56 vars in 1374 bytes} [Fri Jul 1 16:26:36 2022] GET /identity/v3/role_assignments?user.id=dcca9b98066044dca1d7d028b9e83c21&scope.project.id=bf1bfeff7565472da796cac2f99ad6bb&role.id=983be40bdab04c478eaa90c6b6a46b14 => generated 259 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:26:38.071504 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3306fde-5262-435b-9c21-95b7e05a5e93 None None] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:26:38.072060 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3306fde-5262-435b-9c21-95b7e05a5e93 None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:26:38.072060 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3306fde-5262-435b-9c21-95b7e05a5e93 None None] PATH_INFO: `/` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:26:38.074193 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 47/98] 10.222.0.57 () {54 vars in 926 bytes} [Fri Jul 1 16:26:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 16:26:38.079589 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348de088-5c5b-4d86-b7e4-b003d441c7df None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:26:38.080442 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348de088-5c5b-4d86-b7e4-b003d441c7df None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:26:38.080923 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348de088-5c5b-4d86-b7e4-b003d441c7df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:26:38.099231 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-348de088-5c5b-4d86-b7e4-b003d441c7df None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:26:38.100744 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-348de088-5c5b-4d86-b7e4-b003d441c7df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:26:38.161991 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d0dacb-6cd7-480b-b2f9-29703d082e41 None None] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:26:38.162646 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d0dacb-6cd7-480b-b2f9-29703d082e41 None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:26:38.163137 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d0dacb-6cd7-480b-b2f9-29703d082e41 None None] PATH_INFO: `/` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:26:38.164711 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 48/99] 10.222.0.57 () {54 vars in 926 bytes} [Fri Jul 1 16:26:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 01 16:26:38.170028 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f20d995-099c-4f58-a158-a6729238fd3c None None] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:26:38.170580 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f20d995-099c-4f58-a158-a6729238fd3c None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:26:38.171070 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f20d995-099c-4f58-a158-a6729238fd3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:26:38.195038 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.auth.core [None req-3f20d995-099c-4f58-a158-a6729238fd3c None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80312) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:26:38.197680 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-3f20d995-099c-4f58-a158-a6729238fd3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:26:38.226372 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 52/100] 10.222.0.57 () {58 vars in 1023 bytes} [Fri Jul 1 16:26:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 01 16:26:38.270788 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=100 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:26:38.279825 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348de088-5c5b-4d86-b7e4-b003d441c7df None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:26:38.297936 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 49/101] 10.222.0.57 () {58 vars in 1023 bytes} [Fri Jul 1 16:26:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 01 16:26:38.343640 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f20d995-099c-4f58-a158-a6729238fd3c None None] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:26:38.464506 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348de088-5c5b-4d86-b7e4-b003d441c7df None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:26:38.476397 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f20d995-099c-4f58-a158-a6729238fd3c None None] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:26:38.483300 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:26:38.522084 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f83219f-99cb-401a-a0ec-b77ced7cd769 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:26:38.523507 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f83219f-99cb-401a-a0ec-b77ced7cd769 None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:26:38.523964 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f83219f-99cb-401a-a0ec-b77ced7cd769 None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:26:38.524432 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f83219f-99cb-401a-a0ec-b77ced7cd769 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:26:38.536341 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f83219f-99cb-401a-a0ec-b77ced7cd769 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 01 16:26:38.538291 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1ca80e-8069-4648-a99f-858cc60d006a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:26:38.539195 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1ca80e-8069-4648-a99f-858cc60d006a None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:26:38.539560 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1ca80e-8069-4648-a99f-858cc60d006a None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:26:38.539915 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1ca80e-8069-4648-a99f-858cc60d006a None admin] PATH_INFO: `/v3/domains/default` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:26:38.545894 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application [None req-9f83219f-99cb-401a-a0ec-b77ced7cd769 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 01 16:26:38.545894 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 16:26:38.545894 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 16:26:38.545894 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 16:26:38.545894 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 16:26:38.545894 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 16:26:38.545894 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 16:26:38.545894 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 16:26:38.545894 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 16:26:38.545894 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 16:26:38.545894 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 16:26:38.545894 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 16:26:38.545894 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 01 16:26:38.545894 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 01 16:26:38.545894 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 01 16:26:38.545894 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 01 16:26:38.546875 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 01 16:26:38.546875 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 01 16:26:38.546875 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 01 16:26:38.546875 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 01 16:26:38.546875 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 16:26:38.546875 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 16:26:38.546875 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 16:26:38.546875 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 16:26:38.546875 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 16:26:38.546875 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 16:26:38.546875 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 16:26:38.546875 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application with Lock( Jul 01 16:26:38.546875 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 16:26:38.546875 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self._enter() Jul 01 16:26:38.546875 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 16:26:38.546875 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 16:26:38.547670 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 16:26:38.547670 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.creator() Jul 01 16:26:38.547670 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 16:26:38.547670 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application created_value = creator( Jul 01 16:26:38.547670 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 01 16:26:38.547670 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 01 16:26:38.547670 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 01 16:26:38.547670 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 01 16:26:38.547670 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 01 16:26:38.547670 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 01 16:26:38.547670 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 01 16:26:38.547670 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application Jul 01 16:26:38.548266 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c1ca80e-8069-4648-a99f-858cc60d006a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:26:38.549169 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c1ca80e-8069-4648-a99f-858cc60d006a None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:26:38.550424 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 50/102] 10.222.0.57 () {56 vars in 1099 bytes} [Fri Jul 1 16:26:38 2022] GET /identity/v3/roles/reader => generated 84 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 16:26:38.551080 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 53/103] 10.222.0.57 () {56 vars in 1108 bytes} [Fri Jul 1 16:26:38 2022] GET /identity/v3/domains/default => generated 199 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 01 16:26:38.556747 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1ca80e-8069-4648-a99f-858cc60d006a None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:26:38.556994 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f83219f-99cb-401a-a0ec-b77ced7cd769 None admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:26:38.608357 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=100 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:26:38.670851 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f83219f-99cb-401a-a0ec-b77ced7cd769 None admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:26:38.712221 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8275f8-34ce-4b18-b4ee-fee7498d3050 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:26:38.713284 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8275f8-34ce-4b18-b4ee-fee7498d3050 None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:26:38.713764 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8275f8-34ce-4b18-b4ee-fee7498d3050 None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:26:38.714262 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8275f8-34ce-4b18-b4ee-fee7498d3050 None admin] PATH_INFO: `/v3/roles` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:26:38.716419 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e8275f8-34ce-4b18-b4ee-fee7498d3050 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80312) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:26:38.716949 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e8275f8-34ce-4b18-b4ee-fee7498d3050 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:26:38.718555 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e8275f8-34ce-4b18-b4ee-fee7498d3050 None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:26:38.728472 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 51/104] 10.222.0.57 () {56 vars in 1101 bytes} [Fri Jul 1 16:26:38 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:26:38.735124 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8275f8-34ce-4b18-b4ee-fee7498d3050 None admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:26:38.764145 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1ca80e-8069-4648-a99f-858cc60d006a None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:26:38.830846 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f374179-8740-4c9c-b9a3-db9127f24cd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:26:38.832022 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f374179-8740-4c9c-b9a3-db9127f24cd5 None admin] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:26:38.832558 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f374179-8740-4c9c-b9a3-db9127f24cd5 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:26:38.833960 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f374179-8740-4c9c-b9a3-db9127f24cd5 None admin] PATH_INFO: `/v3/projects` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:26:38.834888 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f374179-8740-4c9c-b9a3-db9127f24cd5 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:26:38.836658 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f374179-8740-4c9c-b9a3-db9127f24cd5 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:26:38.883414 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 54/105] 10.222.0.57 () {60 vars in 1141 bytes} [Fri Jul 1 16:26:38 2022] POST /identity/v3/projects => generated 302 bytes in 327 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 01 16:26:38.910032 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8275f8-34ce-4b18-b4ee-fee7498d3050 None admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:26:38.912653 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=100 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:26:38.958309 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ba0ac1-c4fb-4778-b712-0a74ad239195 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:26:38.959234 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ba0ac1-c4fb-4778-b712-0a74ad239195 None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:26:38.959519 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ba0ac1-c4fb-4778-b712-0a74ad239195 None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:26:38.959799 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ba0ac1-c4fb-4778-b712-0a74ad239195 None admin] PATH_INFO: `/v3/users/3d26974a32f240fd853a2dee432b4586` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:26:38.989158 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ba0ac1-c4fb-4778-b712-0a74ad239195 None admin] RBAC: Authorizing `identity:get_user(user_id=3d26974a32f240fd853a2dee432b4586)` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:26:38.990591 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ba0ac1-c4fb-4778-b712-0a74ad239195 None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:26:38.992645 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 52/106] 10.222.0.57 () {56 vars in 1177 bytes} [Fri Jul 1 16:26:38 2022] GET /identity/v3/users/3d26974a32f240fd853a2dee432b4586 => generated 289 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 01 16:26:38.998965 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f374179-8740-4c9c-b9a3-db9127f24cd5 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:26:39.033347 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:26:39.120582 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f374179-8740-4c9c-b9a3-db9127f24cd5 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:26:39.158472 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a52c68-3252-41a5-9e1a-079612c829a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:26:39.159198 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a52c68-3252-41a5-9e1a-079612c829a4 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:26:39.159494 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a52c68-3252-41a5-9e1a-079612c829a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:26:39.159779 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a52c68-3252-41a5-9e1a-079612c829a4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:26:39.197216 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340f74f5-31dd-4b72-8a83-03595dfedd1d None None] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:26:39.197740 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340f74f5-31dd-4b72-8a83-03595dfedd1d None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:26:39.197828 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340f74f5-31dd-4b72-8a83-03595dfedd1d None None] PATH_INFO: `/` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:26:39.199291 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 53/107] 10.222.0.57 () {54 vars in 926 bytes} [Fri Jul 1 16:26:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 16:26:39.204902 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cff07f2-465e-4bad-8a32-9e0b9770dc88 None None] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:26:39.205207 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cff07f2-465e-4bad-8a32-9e0b9770dc88 None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:26:39.205491 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cff07f2-465e-4bad-8a32-9e0b9770dc88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:26:39.235492 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.auth.core [None req-5cff07f2-465e-4bad-8a32-9e0b9770dc88 None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80312) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:26:39.236775 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-5cff07f2-465e-4bad-8a32-9e0b9770dc88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:26:39.253528 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a52c68-3252-41a5-9e1a-079612c829a4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:26:39.254993 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a52c68-3252-41a5-9e1a-079612c829a4 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:26:39.285382 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:26:39.373145 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 54/108] 10.222.0.57 () {58 vars in 1023 bytes} [Fri Jul 1 16:26:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 01 16:26:39.374603 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85865cb-f3cb-4317-a77d-63d33d3440d1 None None] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:26:39.374923 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85865cb-f3cb-4317-a77d-63d33d3440d1 None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:26:39.375236 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85865cb-f3cb-4317-a77d-63d33d3440d1 None None] PATH_INFO: `/` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:26:39.376605 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 55/109] 10.222.0.57 () {54 vars in 926 bytes} [Fri Jul 1 16:26:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 16:26:39.384332 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9383bd2-5cf7-433e-a185-f839922fa640 None None] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:26:39.384649 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9383bd2-5cf7-433e-a185-f839922fa640 None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:26:39.384957 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9383bd2-5cf7-433e-a185-f839922fa640 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:26:39.394540 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 55/110] 10.222.0.57 () {58 vars in 1282 bytes} [Fri Jul 1 16:26:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 396 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 01 16:26:39.401105 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a52c68-3252-41a5-9e1a-079612c829a4 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:26:39.418052 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.auth.core [None req-a9383bd2-5cf7-433e-a185-f839922fa640 None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80312) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:26:39.419393 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-a9383bd2-5cf7-433e-a185-f839922fa640 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:26:39.510644 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 56/111] 10.222.0.57 () {58 vars in 1023 bytes} [Fri Jul 1 16:26:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 01 16:26:39.512746 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9383bd2-5cf7-433e-a185-f839922fa640 None None] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:26:39.546722 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=100 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:26:39.632810 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9383bd2-5cf7-433e-a185-f839922fa640 None None] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:26:39.635261 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a52c68-3252-41a5-9e1a-079612c829a4 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:26:39.672808 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633726b5-c251-4ba6-aa40-b50ce151f05f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:26:39.673625 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633726b5-c251-4ba6-aa40-b50ce151f05f None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:26:39.673924 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633726b5-c251-4ba6-aa40-b50ce151f05f None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:26:39.674214 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633726b5-c251-4ba6-aa40-b50ce151f05f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:26:39.687187 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-633726b5-c251-4ba6-aa40-b50ce151f05f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 01 16:26:39.695586 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application [None req-633726b5-c251-4ba6-aa40-b50ce151f05f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 01 16:26:39.695586 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 16:26:39.695586 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 16:26:39.695586 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 16:26:39.695586 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 16:26:39.695586 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 16:26:39.695586 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 16:26:39.695586 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 16:26:39.695586 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 16:26:39.695586 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 16:26:39.695586 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 16:26:39.695586 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 16:26:39.695586 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 01 16:26:39.695586 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 01 16:26:39.695586 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 01 16:26:39.695586 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 01 16:26:39.696500 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 01 16:26:39.696500 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 01 16:26:39.696500 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 01 16:26:39.696500 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 01 16:26:39.696500 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 16:26:39.696500 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 16:26:39.696500 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 16:26:39.696500 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 16:26:39.696500 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 16:26:39.696500 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 16:26:39.696500 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 16:26:39.696500 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application with Lock( Jul 01 16:26:39.696500 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 16:26:39.696500 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self._enter() Jul 01 16:26:39.696500 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 16:26:39.696500 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 16:26:39.697260 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 16:26:39.697260 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.creator() Jul 01 16:26:39.697260 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 16:26:39.697260 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application created_value = creator( Jul 01 16:26:39.697260 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 01 16:26:39.697260 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 01 16:26:39.697260 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 01 16:26:39.697260 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 01 16:26:39.697260 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 01 16:26:39.697260 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 01 16:26:39.697260 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 01 16:26:39.697260 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application Jul 01 16:26:39.697260 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 57/112] 10.222.0.57 () {56 vars in 1096 bytes} [Fri Jul 1 16:26:39 2022] GET /identity/v3/roles/admin => generated 83 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 16:26:39.697886 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633726b5-c251-4ba6-aa40-b50ce151f05f None admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:26:39.713271 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:26:39.743155 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1e10da-b090-4388-bee8-49d5ca36af48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:26:39.744049 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1e10da-b090-4388-bee8-49d5ca36af48 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:26:39.744338 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1e10da-b090-4388-bee8-49d5ca36af48 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:26:39.744624 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1e10da-b090-4388-bee8-49d5ca36af48 None admin] PATH_INFO: `/v3/projects/5bef204363a04d5d8ceab527b8803f6e` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:26:39.747549 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed1e10da-b090-4388-bee8-49d5ca36af48 None admin] RBAC: Authorizing `identity:get_project(project_id=5bef204363a04d5d8ceab527b8803f6e)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:26:39.749039 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed1e10da-b090-4388-bee8-49d5ca36af48 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:26:39.750991 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 56/113] 10.222.0.57 () {56 vars in 1186 bytes} [Fri Jul 1 16:26:39 2022] GET /identity/v3/projects/5bef204363a04d5d8ceab527b8803f6e => generated 298 bytes in 350 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 01 16:26:39.751610 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1e10da-b090-4388-bee8-49d5ca36af48 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:26:39.810758 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=100 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:26:39.819690 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633726b5-c251-4ba6-aa40-b50ce151f05f None admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:26:39.889425 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b2ceea-9357-4cb6-808b-80ae75623893 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:26:39.890924 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b2ceea-9357-4cb6-808b-80ae75623893 None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:26:39.891383 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b2ceea-9357-4cb6-808b-80ae75623893 None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:26:39.892314 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b2ceea-9357-4cb6-808b-80ae75623893 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:26:39.912830 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b2ceea-9357-4cb6-808b-80ae75623893 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 01 16:26:39.921112 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1e10da-b090-4388-bee8-49d5ca36af48 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:26:39.932263 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application [None req-70b2ceea-9357-4cb6-808b-80ae75623893 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 01 16:26:39.932263 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 16:26:39.932263 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 16:26:39.932263 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 16:26:39.932263 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 16:26:39.932263 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 16:26:39.932263 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 16:26:39.932263 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 16:26:39.932263 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 16:26:39.932263 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 16:26:39.932263 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 16:26:39.932263 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 16:26:39.932263 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 01 16:26:39.932263 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 01 16:26:39.932263 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 01 16:26:39.932263 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 01 16:26:39.934732 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 01 16:26:39.934732 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 01 16:26:39.934732 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 01 16:26:39.934732 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 01 16:26:39.934732 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 16:26:39.934732 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 16:26:39.934732 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 16:26:39.934732 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 16:26:39.934732 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 16:26:39.934732 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 16:26:39.934732 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 16:26:39.934732 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application with Lock( Jul 01 16:26:39.934732 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 16:26:39.934732 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self._enter() Jul 01 16:26:39.934732 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 16:26:39.934732 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 16:26:39.935528 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 16:26:39.935528 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.creator() Jul 01 16:26:39.935528 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 16:26:39.935528 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application created_value = creator( Jul 01 16:26:39.935528 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 01 16:26:39.935528 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 01 16:26:39.935528 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 01 16:26:39.935528 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 01 16:26:39.935528 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 01 16:26:39.935528 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 01 16:26:39.935528 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 01 16:26:39.935528 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application Jul 01 16:26:39.935528 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 58/114] 10.222.0.57 () {56 vars in 1114 bytes} [Fri Jul 1 16:26:39 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 16:26:39.935528 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b2ceea-9357-4cb6-808b-80ae75623893 None admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:26:39.972353 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f775a16d-9a26-4682-95b8-86e2c1a4010e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:26:39.973123 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f775a16d-9a26-4682-95b8-86e2c1a4010e None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:26:39.973455 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f775a16d-9a26-4682-95b8-86e2c1a4010e None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:26:39.973757 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f775a16d-9a26-4682-95b8-86e2c1a4010e None admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:26:39.974613 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f775a16d-9a26-4682-95b8-86e2c1a4010e None admin] RBAC: Adding query filter params (name=admin) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:26:39.974933 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f775a16d-9a26-4682-95b8-86e2c1a4010e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:26:39.976479 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f775a16d-9a26-4682-95b8-86e2c1a4010e None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:26:39.987635 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 57/115] 10.222.0.57 () {56 vars in 1099 bytes} [Fri Jul 1 16:26:39 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:26:39.992177 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bc1e3a-5f55-4ded-9c93-e02e484a4d08 None None] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:26:39.992177 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bc1e3a-5f55-4ded-9c93-e02e484a4d08 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:26:39.992177 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bc1e3a-5f55-4ded-9c93-e02e484a4d08 None None] PATH_INFO: `/` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:26:39.992177 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 58/116] 10.222.0.57 () {54 vars in 926 bytes} [Fri Jul 1 16:26:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 16:26:39.993357 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1bc1e3a-5f55-4ded-9c93-e02e484a4d08 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:26:40.079205 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:26:40.087759 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b2ceea-9357-4cb6-808b-80ae75623893 None admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:26:40.129932 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1bc1e3a-5f55-4ded-9c93-e02e484a4d08 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:26:40.135848 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3cbfbbe-8a76-4b2f-89e8-0262abb43b06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:26:40.136633 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cbfbbe-8a76-4b2f-89e8-0262abb43b06 None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:26:40.136951 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cbfbbe-8a76-4b2f-89e8-0262abb43b06 None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:26:40.137203 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cbfbbe-8a76-4b2f-89e8-0262abb43b06 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:26:40.138066 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3cbfbbe-8a76-4b2f-89e8-0262abb43b06 None admin] RBAC: Adding query filter params (role.id=bd7cefce73d14bd7827b58b6bfbe5e7a, scope.project.id=5bef204363a04d5d8ceab527b8803f6e, user.id=3d26974a32f240fd853a2dee432b4586) {{(pid=80312) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:26:40.138412 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3cbfbbe-8a76-4b2f-89e8-0262abb43b06 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:26:40.139931 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3cbfbbe-8a76-4b2f-89e8-0262abb43b06 None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:26:40.148283 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 59/117] 10.222.0.57 () {56 vars in 1374 bytes} [Fri Jul 1 16:26:39 2022] GET /identity/v3/role_assignments?user.id=3d26974a32f240fd853a2dee432b4586&scope.project.id=5bef204363a04d5d8ceab527b8803f6e&role.id=bd7cefce73d14bd7827b58b6bfbe5e7a => generated 259 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:26:40.149786 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3cbfbbe-8a76-4b2f-89e8-0262abb43b06 None admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:26:40.163069 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=100 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:26:40.175412 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5177014b-2cc0-45d1-aa51-790d70875ec8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:26:40.176674 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5177014b-2cc0-45d1-aa51-790d70875ec8 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:26:40.177441 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5177014b-2cc0-45d1-aa51-790d70875ec8 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:26:40.178079 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5177014b-2cc0-45d1-aa51-790d70875ec8 None admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:26:40.179200 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5177014b-2cc0-45d1-aa51-790d70875ec8 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:26:40.179718 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5177014b-2cc0-45d1-aa51-790d70875ec8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:26:40.181741 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5177014b-2cc0-45d1-aa51-790d70875ec8 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:26:40.195666 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 59/118] 10.222.0.57 () {56 vars in 1111 bytes} [Fri Jul 1 16:26:39 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:26:40.197616 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096f60bb-ca6c-4494-8677-fe234e51134e None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:26:40.198142 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096f60bb-ca6c-4494-8677-fe234e51134e None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:26:40.199261 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096f60bb-ca6c-4494-8677-fe234e51134e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:26:40.223095 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-096f60bb-ca6c-4494-8677-fe234e51134e None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:26:40.224964 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-096f60bb-ca6c-4494-8677-fe234e51134e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:26:40.318191 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3cbfbbe-8a76-4b2f-89e8-0262abb43b06 None admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:26:40.331761 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 60/119] 10.222.0.57 () {58 vars in 1023 bytes} [Fri Jul 1 16:26:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 01 16:26:40.334381 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d1de0d-298f-4c55-b683-6315609839f8 None None] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:26:40.335132 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d1de0d-298f-4c55-b683-6315609839f8 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:26:40.335965 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d1de0d-298f-4c55-b683-6315609839f8 None None] PATH_INFO: `/` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:26:40.338685 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 61/120] 10.222.0.57 () {54 vars in 926 bytes} [Fri Jul 1 16:26:40 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 16:26:40.340763 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d1de0d-298f-4c55-b683-6315609839f8 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:26:40.391511 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f06483-8b65-44b9-8933-d5932ed0859f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:26:40.392535 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f06483-8b65-44b9-8933-d5932ed0859f None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:26:40.392924 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f06483-8b65-44b9-8933-d5932ed0859f None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:26:40.393210 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f06483-8b65-44b9-8933-d5932ed0859f None admin] PATH_INFO: `/v3/users/9996f66c673f4673ab2342b66da9acc2` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:26:40.394509 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f06483-8b65-44b9-8933-d5932ed0859f None admin] RBAC: Authorizing `identity:get_user(user_id=9996f66c673f4673ab2342b66da9acc2)` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:26:40.395984 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f06483-8b65-44b9-8933-d5932ed0859f None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:26:40.397678 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 60/121] 10.222.0.57 () {56 vars in 1177 bytes} [Fri Jul 1 16:26:40 2022] GET /identity/v3/users/9996f66c673f4673ab2342b66da9acc2 => generated 247 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:26:40.398936 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35731fc9-10c3-4a10-bc6b-e5904099a186 None None] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:26:40.399227 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35731fc9-10c3-4a10-bc6b-e5904099a186 None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:26:40.399510 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35731fc9-10c3-4a10-bc6b-e5904099a186 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:26:40.418068 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.auth.core [None req-35731fc9-10c3-4a10-bc6b-e5904099a186 None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80312) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:26:40.419350 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-35731fc9-10c3-4a10-bc6b-e5904099a186 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:26:40.436955 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:26:40.502168 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 61/122] 10.222.0.57 () {58 vars in 1023 bytes} [Fri Jul 1 16:26:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 01 16:26:40.507348 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35731fc9-10c3-4a10-bc6b-e5904099a186 None None] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:26:40.515413 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=100 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:26:40.556509 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d1de0d-298f-4c55-b683-6315609839f8 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:26:40.595932 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5769fc5b-9260-400c-bc75-4004a13c02aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:26:40.596783 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5769fc5b-9260-400c-bc75-4004a13c02aa None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:26:40.597154 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5769fc5b-9260-400c-bc75-4004a13c02aa None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:26:40.597534 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5769fc5b-9260-400c-bc75-4004a13c02aa None admin] PATH_INFO: `/v3/users/dcca9b98066044dca1d7d028b9e83c21` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:26:40.599010 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5769fc5b-9260-400c-bc75-4004a13c02aa None admin] RBAC: Authorizing `identity:get_user(user_id=dcca9b98066044dca1d7d028b9e83c21)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:26:40.600703 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5769fc5b-9260-400c-bc75-4004a13c02aa None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:26:40.602863 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 62/123] 10.222.0.57 () {56 vars in 1177 bytes} [Fri Jul 1 16:26:40 2022] GET /identity/v3/users/dcca9b98066044dca1d7d028b9e83c21 => generated 275 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:26:40.608717 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5769fc5b-9260-400c-bc75-4004a13c02aa None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:26:40.652146 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35731fc9-10c3-4a10-bc6b-e5904099a186 None None] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:26:40.692898 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae934a5-eeec-4f13-b164-52c14693229e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:26:40.693712 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae934a5-eeec-4f13-b164-52c14693229e None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:26:40.694000 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae934a5-eeec-4f13-b164-52c14693229e None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:26:40.694288 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae934a5-eeec-4f13-b164-52c14693229e None admin] PATH_INFO: `/v3/roles/member` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:26:40.706765 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae934a5-eeec-4f13-b164-52c14693229e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 01 16:26:40.715297 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application [None req-eae934a5-eeec-4f13-b164-52c14693229e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 01 16:26:40.715297 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 16:26:40.715297 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 16:26:40.715297 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 16:26:40.715297 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 16:26:40.715297 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 16:26:40.715297 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 16:26:40.715297 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 16:26:40.715297 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 16:26:40.715297 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 16:26:40.715297 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 16:26:40.715297 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 16:26:40.715297 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 01 16:26:40.715297 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 01 16:26:40.715297 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 01 16:26:40.715297 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 01 16:26:40.716153 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 01 16:26:40.716153 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 01 16:26:40.716153 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 01 16:26:40.716153 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 01 16:26:40.716153 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 16:26:40.716153 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 16:26:40.716153 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 16:26:40.716153 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 16:26:40.716153 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 16:26:40.716153 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 16:26:40.716153 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 16:26:40.716153 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application with Lock( Jul 01 16:26:40.716153 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 16:26:40.716153 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self._enter() Jul 01 16:26:40.716153 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 16:26:40.716153 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 16:26:40.717170 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 16:26:40.717170 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.creator() Jul 01 16:26:40.717170 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 16:26:40.717170 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application created_value = creator( Jul 01 16:26:40.717170 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 01 16:26:40.717170 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 01 16:26:40.717170 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 01 16:26:40.717170 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 01 16:26:40.717170 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 01 16:26:40.717170 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 01 16:26:40.717170 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 01 16:26:40.717170 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application Jul 01 16:26:40.717170 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 62/124] 10.222.0.57 () {56 vars in 1099 bytes} [Fri Jul 1 16:26:40 2022] GET /identity/v3/roles/member => generated 84 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 16:26:40.722382 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae934a5-eeec-4f13-b164-52c14693229e None admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:26:40.730111 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5769fc5b-9260-400c-bc75-4004a13c02aa None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:26:40.752697 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:26:40.772777 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c6052e-5d78-4d67-806a-0df993dee8b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:26:40.773780 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c6052e-5d78-4d67-806a-0df993dee8b8 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:26:40.774167 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c6052e-5d78-4d67-806a-0df993dee8b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:26:40.774582 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c6052e-5d78-4d67-806a-0df993dee8b8 None admin] PATH_INFO: `/v3/projects/5bef204363a04d5d8ceab527b8803f6e` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:26:40.776293 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c6052e-5d78-4d67-806a-0df993dee8b8 None admin] RBAC: Authorizing `identity:get_project(project_id=5bef204363a04d5d8ceab527b8803f6e)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:26:40.778054 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c6052e-5d78-4d67-806a-0df993dee8b8 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:26:40.780035 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 63/125] 10.222.0.57 () {56 vars in 1186 bytes} [Fri Jul 1 16:26:40 2022] GET /identity/v3/projects/5bef204363a04d5d8ceab527b8803f6e => generated 298 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:26:40.785590 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c6052e-5d78-4d67-806a-0df993dee8b8 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:26:40.802071 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=100 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:26:40.858383 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae934a5-eeec-4f13-b164-52c14693229e None admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:26:40.901431 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c6052e-5d78-4d67-806a-0df993dee8b8 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:26:40.902544 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544d5515-0aeb-4d7a-8029-252d6bd563ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:26:40.903612 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544d5515-0aeb-4d7a-8029-252d6bd563ae None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:26:40.903911 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544d5515-0aeb-4d7a-8029-252d6bd563ae None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:26:40.904200 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544d5515-0aeb-4d7a-8029-252d6bd563ae None admin] PATH_INFO: `/v3/roles/member` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:26:40.919294 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-544d5515-0aeb-4d7a-8029-252d6bd563ae None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 01 16:26:40.929949 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application [None req-544d5515-0aeb-4d7a-8029-252d6bd563ae None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 01 16:26:40.929949 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 16:26:40.929949 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 16:26:40.929949 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 16:26:40.929949 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 16:26:40.929949 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 16:26:40.929949 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 16:26:40.929949 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 16:26:40.929949 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 16:26:40.929949 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 16:26:40.929949 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 16:26:40.929949 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 16:26:40.929949 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 01 16:26:40.929949 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 01 16:26:40.929949 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 01 16:26:40.929949 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 01 16:26:40.931003 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 01 16:26:40.931003 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 01 16:26:40.931003 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 01 16:26:40.931003 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 01 16:26:40.931003 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 16:26:40.931003 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 16:26:40.931003 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 16:26:40.931003 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 16:26:40.931003 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 16:26:40.931003 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 16:26:40.931003 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 16:26:40.931003 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application with Lock( Jul 01 16:26:40.931003 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 16:26:40.931003 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self._enter() Jul 01 16:26:40.931003 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 16:26:40.931003 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 16:26:40.931765 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 16:26:40.931765 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.creator() Jul 01 16:26:40.931765 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 16:26:40.931765 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application created_value = creator( Jul 01 16:26:40.931765 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 01 16:26:40.931765 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 01 16:26:40.931765 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 01 16:26:40.931765 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 01 16:26:40.931765 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 01 16:26:40.931765 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 01 16:26:40.931765 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 01 16:26:40.931765 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application Jul 01 16:26:40.933789 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 63/126] 10.222.0.57 () {56 vars in 1099 bytes} [Fri Jul 1 16:26:40 2022] GET /identity/v3/roles/member => generated 84 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 16:26:40.933789 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544d5515-0aeb-4d7a-8029-252d6bd563ae None admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:26:40.944564 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0acdc3-5149-4db5-bf5a-e68c77037c6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:26:40.945558 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0acdc3-5149-4db5-bf5a-e68c77037c6a None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:26:40.945941 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0acdc3-5149-4db5-bf5a-e68c77037c6a None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:26:40.946305 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0acdc3-5149-4db5-bf5a-e68c77037c6a None admin] PATH_INFO: `/v3/projects/5bef204363a04d5d8ceab527b8803f6e` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:26:40.947589 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0acdc3-5149-4db5-bf5a-e68c77037c6a None admin] RBAC: Authorizing `identity:get_project(project_id=5bef204363a04d5d8ceab527b8803f6e)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:26:40.949692 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0acdc3-5149-4db5-bf5a-e68c77037c6a None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:26:40.951629 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 64/127] 10.222.0.57 () {56 vars in 1186 bytes} [Fri Jul 1 16:26:40 2022] GET /identity/v3/projects/5bef204363a04d5d8ceab527b8803f6e => generated 298 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:26:40.952429 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0acdc3-5149-4db5-bf5a-e68c77037c6a None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:26:41.014751 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:26:41.067440 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0acdc3-5149-4db5-bf5a-e68c77037c6a None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:26:41.078667 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544d5515-0aeb-4d7a-8029-252d6bd563ae None admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:26:41.103206 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=100 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:26:41.115998 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9eb55b-bc7b-48c6-942b-9788e773622f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:26:41.116902 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9eb55b-bc7b-48c6-942b-9788e773622f None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:26:41.117426 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9eb55b-bc7b-48c6-942b-9788e773622f None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:26:41.117804 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9eb55b-bc7b-48c6-942b-9788e773622f None admin] PATH_INFO: `/v3/roles` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:26:41.118387 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a977dd-ea3c-45fa-9c95-e0683a9deb71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:26:41.118791 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e9eb55b-bc7b-48c6-942b-9788e773622f None admin] RBAC: Adding query filter params (name=member) {{(pid=80312) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:26:41.119198 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e9eb55b-bc7b-48c6-942b-9788e773622f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:26:41.119308 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a977dd-ea3c-45fa-9c95-e0683a9deb71 None admin] REQUEST_METHOD: `PUT` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:26:41.119608 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a977dd-ea3c-45fa-9c95-e0683a9deb71 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:26:41.119894 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a977dd-ea3c-45fa-9c95-e0683a9deb71 None admin] PATH_INFO: `/v3/projects/5bef204363a04d5d8ceab527b8803f6e/users/9996f66c673f4673ab2342b66da9acc2/roles/84ee739146934c1ab32c501ff8093644` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:26:41.121018 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e9eb55b-bc7b-48c6-942b-9788e773622f None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:26:41.122314 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a977dd-ea3c-45fa-9c95-e0683a9deb71 None admin] RBAC: Authorizing `identity:create_grant(project_id=5bef204363a04d5d8ceab527b8803f6e, user_id=9996f66c673f4673ab2342b66da9acc2, role_id=84ee739146934c1ab32c501ff8093644)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:26:41.124534 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a977dd-ea3c-45fa-9c95-e0683a9deb71 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:26:41.130218 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 64/128] 10.222.0.57 () {56 vars in 1101 bytes} [Fri Jul 1 16:26:40 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:26:41.131333 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9eb55b-bc7b-48c6-942b-9788e773622f None admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:26:41.147322 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 65/129] 10.222.0.57 () {58 vars in 1439 bytes} [Fri Jul 1 16:26:40 2022] PUT /identity/v3/projects/5bef204363a04d5d8ceab527b8803f6e/users/9996f66c673f4673ab2342b66da9acc2/roles/84ee739146934c1ab32c501ff8093644 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 16:26:41.149660 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a977dd-ea3c-45fa-9c95-e0683a9deb71 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:26:41.355428 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9eb55b-bc7b-48c6-942b-9788e773622f None admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:26:41.360472 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a977dd-ea3c-45fa-9c95-e0683a9deb71 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:26:41.377149 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:26:41.407077 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc6e64d-0ccc-4930-ac1c-11c99744e737 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:26:41.407845 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc6e64d-0ccc-4930-ac1c-11c99744e737 None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:26:41.408043 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38420bf9-f8d3-466d-8c4d-922d03a5b471 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:26:41.408181 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc6e64d-0ccc-4930-ac1c-11c99744e737 None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:26:41.408549 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc6e64d-0ccc-4930-ac1c-11c99744e737 None admin] PATH_INFO: `/v3/roles` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:26:41.409451 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc6e64d-0ccc-4930-ac1c-11c99744e737 None admin] RBAC: Adding query filter params (name=member) {{(pid=80312) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:26:41.409676 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38420bf9-f8d3-466d-8c4d-922d03a5b471 None admin] REQUEST_METHOD: `PUT` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:26:41.409880 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc6e64d-0ccc-4930-ac1c-11c99744e737 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:26:41.410408 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38420bf9-f8d3-466d-8c4d-922d03a5b471 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:26:41.410927 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38420bf9-f8d3-466d-8c4d-922d03a5b471 None admin] PATH_INFO: `/v3/projects/5bef204363a04d5d8ceab527b8803f6e/users/dcca9b98066044dca1d7d028b9e83c21/roles/a8e5c73da81d4f0eb7b92c7058e500fe` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:26:41.411496 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc6e64d-0ccc-4930-ac1c-11c99744e737 None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:26:41.414412 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38420bf9-f8d3-466d-8c4d-922d03a5b471 None admin] RBAC: Authorizing `identity:create_grant(project_id=5bef204363a04d5d8ceab527b8803f6e, user_id=dcca9b98066044dca1d7d028b9e83c21, role_id=a8e5c73da81d4f0eb7b92c7058e500fe)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:26:41.416209 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38420bf9-f8d3-466d-8c4d-922d03a5b471 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:26:41.421501 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 65/130] 10.222.0.57 () {56 vars in 1101 bytes} [Fri Jul 1 16:26:41 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:26:41.426379 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc6e64d-0ccc-4930-ac1c-11c99744e737 None admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:26:41.453931 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 66/131] 10.222.0.57 () {58 vars in 1439 bytes} [Fri Jul 1 16:26:41 2022] PUT /identity/v3/projects/5bef204363a04d5d8ceab527b8803f6e/users/dcca9b98066044dca1d7d028b9e83c21/roles/a8e5c73da81d4f0eb7b92c7058e500fe => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 16:26:41.456909 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38420bf9-f8d3-466d-8c4d-922d03a5b471 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:26:41.544142 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=100 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:26:41.649382 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38420bf9-f8d3-466d-8c4d-922d03a5b471 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:26:41.680353 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc6e64d-0ccc-4930-ac1c-11c99744e737 None admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:26:41.690716 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0650854c-5299-4c2f-b5ca-a1f5054f08cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:26:41.691598 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0650854c-5299-4c2f-b5ca-a1f5054f08cf None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:26:41.691884 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0650854c-5299-4c2f-b5ca-a1f5054f08cf None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:26:41.692167 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0650854c-5299-4c2f-b5ca-a1f5054f08cf None admin] PATH_INFO: `/v3/users/dcca9b98066044dca1d7d028b9e83c21` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:26:41.693858 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0650854c-5299-4c2f-b5ca-a1f5054f08cf None admin] RBAC: Authorizing `identity:get_user(user_id=dcca9b98066044dca1d7d028b9e83c21)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:26:41.695351 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0650854c-5299-4c2f-b5ca-a1f5054f08cf None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:26:41.697360 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 67/132] 10.222.0.57 () {56 vars in 1177 bytes} [Fri Jul 1 16:26:41 2022] GET /identity/v3/users/dcca9b98066044dca1d7d028b9e83c21 => generated 275 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:26:41.699395 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3d6565-f3a9-4392-b108-8d8bfd267b0c None None] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:26:41.699685 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3d6565-f3a9-4392-b108-8d8bfd267b0c None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:26:41.705139 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3d6565-f3a9-4392-b108-8d8bfd267b0c None None] PATH_INFO: `/` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:26:41.707479 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 68/133] 10.222.0.57 () {54 vars in 926 bytes} [Fri Jul 1 16:26:41 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 16:26:41.708126 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3d6565-f3a9-4392-b108-8d8bfd267b0c None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:26:41.758209 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:26:41.775126 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01fd098-20f9-46bf-8df0-e424afdc52af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:26:41.776069 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01fd098-20f9-46bf-8df0-e424afdc52af None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:26:41.776361 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01fd098-20f9-46bf-8df0-e424afdc52af None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:26:41.777238 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01fd098-20f9-46bf-8df0-e424afdc52af None admin] PATH_INFO: `/v3/users/dcca9b98066044dca1d7d028b9e83c21` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:26:41.778732 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a01fd098-20f9-46bf-8df0-e424afdc52af None admin] RBAC: Authorizing `identity:get_user(user_id=dcca9b98066044dca1d7d028b9e83c21)` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:26:41.780323 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a01fd098-20f9-46bf-8df0-e424afdc52af None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:26:41.782218 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 66/134] 10.222.0.57 () {56 vars in 1177 bytes} [Fri Jul 1 16:26:41 2022] GET /identity/v3/users/dcca9b98066044dca1d7d028b9e83c21 => generated 275 bytes in 360 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:26:41.783413 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea93d57-cac0-4478-af18-47e5646a3107 None None] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:26:41.783702 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea93d57-cac0-4478-af18-47e5646a3107 None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:26:41.783982 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea93d57-cac0-4478-af18-47e5646a3107 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:26:41.801625 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.auth.core [None req-3ea93d57-cac0-4478-af18-47e5646a3107 None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80312) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:26:41.802870 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-3ea93d57-cac0-4478-af18-47e5646a3107 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:26:41.836958 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3d6565-f3a9-4392-b108-8d8bfd267b0c None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:26:41.886962 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 67/135] 10.222.0.57 () {58 vars in 1023 bytes} [Fri Jul 1 16:26:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 01 16:26:41.888171 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea93d57-cac0-4478-af18-47e5646a3107 None None] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:26:41.890012 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b813682-3712-4837-bace-8f75c9950e7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:26:41.891049 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b813682-3712-4837-bace-8f75c9950e7f None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:26:41.891376 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b813682-3712-4837-bace-8f75c9950e7f None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:26:41.891654 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b813682-3712-4837-bace-8f75c9950e7f None admin] PATH_INFO: `/v3/projects/bf1bfeff7565472da796cac2f99ad6bb` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:26:41.892977 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b813682-3712-4837-bace-8f75c9950e7f None admin] RBAC: Authorizing `identity:get_project(project_id=bf1bfeff7565472da796cac2f99ad6bb)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:26:41.899377 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b813682-3712-4837-bace-8f75c9950e7f None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:26:41.902305 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 69/136] 10.222.0.57 () {56 vars in 1186 bytes} [Fri Jul 1 16:26:41 2022] GET /identity/v3/projects/bf1bfeff7565472da796cac2f99ad6bb => generated 312 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:26:41.907991 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b813682-3712-4837-bace-8f75c9950e7f None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:26:41.942643 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=100 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:26:42.027534 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea93d57-cac0-4478-af18-47e5646a3107 None None] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:26:42.106748 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5674f1a5-64d8-41fa-97d8-47e3fdb5ee50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:26:42.108022 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5674f1a5-64d8-41fa-97d8-47e3fdb5ee50 None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:26:42.110067 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b813682-3712-4837-bace-8f75c9950e7f None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:26:42.110684 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5674f1a5-64d8-41fa-97d8-47e3fdb5ee50 None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:26:42.111170 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5674f1a5-64d8-41fa-97d8-47e3fdb5ee50 None admin] PATH_INFO: `/v3/projects/5bef204363a04d5d8ceab527b8803f6e` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:26:42.114070 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5674f1a5-64d8-41fa-97d8-47e3fdb5ee50 None admin] RBAC: Authorizing `identity:get_project(project_id=5bef204363a04d5d8ceab527b8803f6e)` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:26:42.115791 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5674f1a5-64d8-41fa-97d8-47e3fdb5ee50 None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:26:42.119718 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 68/137] 10.222.0.57 () {56 vars in 1186 bytes} [Fri Jul 1 16:26:41 2022] GET /identity/v3/projects/5bef204363a04d5d8ceab527b8803f6e => generated 298 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:26:42.126188 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5674f1a5-64d8-41fa-97d8-47e3fdb5ee50 None admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:26:42.140550 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:26:42.189935 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ad06ef-7efd-4bb4-91ae-d272beb4fc18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:26:42.191636 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ad06ef-7efd-4bb4-91ae-d272beb4fc18 None admin] REQUEST_METHOD: `PUT` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:26:42.192126 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ad06ef-7efd-4bb4-91ae-d272beb4fc18 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:26:42.192606 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ad06ef-7efd-4bb4-91ae-d272beb4fc18 None admin] PATH_INFO: `/v3/projects/bf1bfeff7565472da796cac2f99ad6bb/users/dcca9b98066044dca1d7d028b9e83c21/roles/983be40bdab04c478eaa90c6b6a46b14` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:26:42.194969 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ad06ef-7efd-4bb4-91ae-d272beb4fc18 None admin] RBAC: Authorizing `identity:create_grant(project_id=bf1bfeff7565472da796cac2f99ad6bb, user_id=dcca9b98066044dca1d7d028b9e83c21, role_id=983be40bdab04c478eaa90c6b6a46b14)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:26:42.197190 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ad06ef-7efd-4bb4-91ae-d272beb4fc18 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:26:42.217705 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 70/138] 10.222.0.57 () {58 vars in 1439 bytes} [Fri Jul 1 16:26:41 2022] PUT /identity/v3/projects/bf1bfeff7565472da796cac2f99ad6bb/users/dcca9b98066044dca1d7d028b9e83c21/roles/983be40bdab04c478eaa90c6b6a46b14 => generated 0 bytes in 310 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 16:26:42.219039 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ad06ef-7efd-4bb4-91ae-d272beb4fc18 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:26:42.373870 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5674f1a5-64d8-41fa-97d8-47e3fdb5ee50 None admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:26:42.399326 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=100 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:26:42.415487 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9746741b-1490-4a48-afd5-54727322dfcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:26:42.416593 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9746741b-1490-4a48-afd5-54727322dfcd None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:26:42.416990 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9746741b-1490-4a48-afd5-54727322dfcd None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:26:42.417938 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9746741b-1490-4a48-afd5-54727322dfcd None admin] PATH_INFO: `/v3/domains/default` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:26:42.419879 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9746741b-1490-4a48-afd5-54727322dfcd None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:26:42.421511 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9746741b-1490-4a48-afd5-54727322dfcd None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:26:42.423600 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 69/139] 10.222.0.57 () {56 vars in 1108 bytes} [Fri Jul 1 16:26:42 2022] GET /identity/v3/domains/default => generated 199 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:26:42.429286 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9746741b-1490-4a48-afd5-54727322dfcd None admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:26:42.430068 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ad06ef-7efd-4bb4-91ae-d272beb4fc18 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:26:42.499930 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5caa005f-a235-43f2-9803-4c49059c8f1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:26:42.500775 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5caa005f-a235-43f2-9803-4c49059c8f1a None admin] REQUEST_METHOD: `PUT` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:26:42.501062 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5caa005f-a235-43f2-9803-4c49059c8f1a None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:26:42.501342 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5caa005f-a235-43f2-9803-4c49059c8f1a None admin] PATH_INFO: `/v3/projects/5bef204363a04d5d8ceab527b8803f6e/users/dcca9b98066044dca1d7d028b9e83c21/roles/983be40bdab04c478eaa90c6b6a46b14` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:26:42.507940 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5caa005f-a235-43f2-9803-4c49059c8f1a None admin] RBAC: Authorizing `identity:create_grant(project_id=5bef204363a04d5d8ceab527b8803f6e, user_id=dcca9b98066044dca1d7d028b9e83c21, role_id=983be40bdab04c478eaa90c6b6a46b14)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:26:42.509416 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5caa005f-a235-43f2-9803-4c49059c8f1a None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:26:42.540426 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 71/140] 10.222.0.57 () {58 vars in 1439 bytes} [Fri Jul 1 16:26:42 2022] PUT /identity/v3/projects/5bef204363a04d5d8ceab527b8803f6e/users/dcca9b98066044dca1d7d028b9e83c21/roles/983be40bdab04c478eaa90c6b6a46b14 => generated 0 bytes in 322 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 16:26:42.546615 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9746741b-1490-4a48-afd5-54727322dfcd None admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:26:42.586595 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f1e55c-5add-414e-a534-d51d72cb4ce4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:26:42.587324 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f1e55c-5add-414e-a534-d51d72cb4ce4 None admin] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:26:42.587674 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f1e55c-5add-414e-a534-d51d72cb4ce4 None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:26:42.588023 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f1e55c-5add-414e-a534-d51d72cb4ce4 None admin] PATH_INFO: `/v3/users` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:26:42.588786 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f1e55c-5add-414e-a534-d51d72cb4ce4 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:26:42.591338 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f1e55c-5add-414e-a534-d51d72cb4ce4 None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:26:42.617062 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 70/141] 10.222.0.57 () {60 vars in 1132 bytes} [Fri Jul 1 16:26:42 2022] POST /identity/v3/users => generated 283 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 01 16:26:42.765560 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdf054a-39c0-41aa-a7ed-869eedbd2c09 None None] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:26:42.765878 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdf054a-39c0-41aa-a7ed-869eedbd2c09 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:26:42.766059 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdf054a-39c0-41aa-a7ed-869eedbd2c09 None None] PATH_INFO: `/` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:26:42.767874 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 72/142] 10.222.0.57 () {54 vars in 926 bytes} [Fri Jul 1 16:26:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 16:26:42.773247 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90ab54b-fbe7-4752-aef6-1b6026cd10ed None None] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:26:42.773547 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90ab54b-fbe7-4752-aef6-1b6026cd10ed None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:26:42.773844 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90ab54b-fbe7-4752-aef6-1b6026cd10ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:26:42.793673 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.auth.core [None req-a90ab54b-fbe7-4752-aef6-1b6026cd10ed None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80312) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:26:42.794947 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-a90ab54b-fbe7-4752-aef6-1b6026cd10ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:26:42.830212 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:26:42.892301 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 71/143] 10.222.0.57 () {58 vars in 1023 bytes} [Fri Jul 1 16:26:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 01 16:26:42.916129 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cdf054a-39c0-41aa-a7ed-869eedbd2c09 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:26:42.967962 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:26:43.062680 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cdf054a-39c0-41aa-a7ed-869eedbd2c09 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:26:43.114966 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf38740-eb29-46dc-9301-afe667a671f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:26:43.116167 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf38740-eb29-46dc-9301-afe667a671f2 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:26:43.116668 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf38740-eb29-46dc-9301-afe667a671f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:26:43.117146 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf38740-eb29-46dc-9301-afe667a671f2 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:26:43.132127 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf38740-eb29-46dc-9301-afe667a671f2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 01 16:26:43.140926 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application [None req-ebf38740-eb29-46dc-9301-afe667a671f2 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 01 16:26:43.140926 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 16:26:43.140926 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 16:26:43.140926 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 16:26:43.140926 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 16:26:43.140926 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 16:26:43.140926 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 16:26:43.140926 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 16:26:43.140926 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 16:26:43.140926 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 16:26:43.140926 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 16:26:43.140926 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 16:26:43.140926 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 01 16:26:43.140926 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 01 16:26:43.140926 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 01 16:26:43.140926 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 01 16:26:43.141810 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 01 16:26:43.141810 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 01 16:26:43.141810 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 01 16:26:43.141810 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 01 16:26:43.141810 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 16:26:43.141810 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 16:26:43.141810 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 16:26:43.141810 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 16:26:43.141810 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 16:26:43.141810 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 16:26:43.141810 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 16:26:43.141810 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application with Lock( Jul 01 16:26:43.141810 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 16:26:43.141810 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._enter() Jul 01 16:26:43.141810 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 16:26:43.141810 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 16:26:43.142614 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 16:26:43.142614 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.creator() Jul 01 16:26:43.142614 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 16:26:43.142614 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application created_value = creator( Jul 01 16:26:43.142614 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 01 16:26:43.142614 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 01 16:26:43.142614 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 01 16:26:43.142614 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 01 16:26:43.142614 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 01 16:26:43.142614 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 01 16:26:43.142614 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 01 16:26:43.142614 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Jul 01 16:26:43.145031 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 73/144] 10.222.0.57 () {56 vars in 1099 bytes} [Fri Jul 1 16:26:42 2022] GET /identity/v3/roles/reader => generated 84 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 16:26:43.152446 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90ab54b-fbe7-4752-aef6-1b6026cd10ed None None] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:26:43.305047 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90ab54b-fbe7-4752-aef6-1b6026cd10ed None None] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:26:43.354149 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ae2c77-cd00-4086-8112-70a258415086 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:26:43.355310 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ae2c77-cd00-4086-8112-70a258415086 None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:26:43.355576 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ae2c77-cd00-4086-8112-70a258415086 None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:26:43.355860 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ae2c77-cd00-4086-8112-70a258415086 None admin] PATH_INFO: `/v3/roles` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:26:43.356633 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ae2c77-cd00-4086-8112-70a258415086 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80312) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:26:43.356946 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ae2c77-cd00-4086-8112-70a258415086 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:26:43.358367 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ae2c77-cd00-4086-8112-70a258415086 None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:26:43.367285 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 72/145] 10.222.0.57 () {56 vars in 1101 bytes} [Fri Jul 1 16:26:43 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:26:43.373305 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf38740-eb29-46dc-9301-afe667a671f2 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:26:43.486892 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:26:43.501974 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf38740-eb29-46dc-9301-afe667a671f2 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:26:43.540869 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c89a7bf-ebe3-46bb-afba-ce5a615cc752 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:26:43.541754 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c89a7bf-ebe3-46bb-afba-ce5a615cc752 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:26:43.542087 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c89a7bf-ebe3-46bb-afba-ce5a615cc752 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:26:43.542472 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c89a7bf-ebe3-46bb-afba-ce5a615cc752 None admin] PATH_INFO: `/v3/users/3d26974a32f240fd853a2dee432b4586` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:26:43.547060 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c89a7bf-ebe3-46bb-afba-ce5a615cc752 None admin] RBAC: Authorizing `identity:get_user(user_id=3d26974a32f240fd853a2dee432b4586)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:26:43.548532 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c89a7bf-ebe3-46bb-afba-ce5a615cc752 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:26:43.550539 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 74/146] 10.222.0.57 () {56 vars in 1177 bytes} [Fri Jul 1 16:26:43 2022] GET /identity/v3/users/3d26974a32f240fd853a2dee432b4586 => generated 289 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:26:43.556011 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ae2c77-cd00-4086-8112-70a258415086 None admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:26:43.594318 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=100 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:26:43.686198 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ae2c77-cd00-4086-8112-70a258415086 None admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:26:43.730409 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25cb1e2-6e10-4acf-b514-c2caf951088f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:26:43.731366 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25cb1e2-6e10-4acf-b514-c2caf951088f None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:26:43.731656 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25cb1e2-6e10-4acf-b514-c2caf951088f None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:26:43.731945 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25cb1e2-6e10-4acf-b514-c2caf951088f None admin] PATH_INFO: `/v3/projects/5bef204363a04d5d8ceab527b8803f6e` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:26:43.733304 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25cb1e2-6e10-4acf-b514-c2caf951088f None admin] RBAC: Authorizing `identity:get_project(project_id=5bef204363a04d5d8ceab527b8803f6e)` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:26:43.734815 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25cb1e2-6e10-4acf-b514-c2caf951088f None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:26:43.736742 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 73/147] 10.222.0.57 () {56 vars in 1186 bytes} [Fri Jul 1 16:26:43 2022] GET /identity/v3/projects/5bef204363a04d5d8ceab527b8803f6e => generated 298 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:26:43.742669 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c89a7bf-ebe3-46bb-afba-ce5a615cc752 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:26:43.892819 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c89a7bf-ebe3-46bb-afba-ce5a615cc752 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:26:43.941099 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2e92d3-673c-47f0-a0c1-c95cdd228d89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:26:43.941938 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2e92d3-673c-47f0-a0c1-c95cdd228d89 None admin] REQUEST_METHOD: `PUT` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:26:43.942215 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2e92d3-673c-47f0-a0c1-c95cdd228d89 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:26:43.942522 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2e92d3-673c-47f0-a0c1-c95cdd228d89 None admin] PATH_INFO: `/v3/projects/5bef204363a04d5d8ceab527b8803f6e/users/3d26974a32f240fd853a2dee432b4586/roles/bd7cefce73d14bd7827b58b6bfbe5e7a` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:26:43.944573 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f2e92d3-673c-47f0-a0c1-c95cdd228d89 None admin] RBAC: Authorizing `identity:create_grant(project_id=5bef204363a04d5d8ceab527b8803f6e, user_id=3d26974a32f240fd853a2dee432b4586, role_id=bd7cefce73d14bd7827b58b6bfbe5e7a)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:26:43.946023 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f2e92d3-673c-47f0-a0c1-c95cdd228d89 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:26:43.962788 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 75/148] 10.222.0.57 () {58 vars in 1439 bytes} [Fri Jul 1 16:26:43 2022] PUT /identity/v3/projects/5bef204363a04d5d8ceab527b8803f6e/users/3d26974a32f240fd853a2dee432b4586/roles/bd7cefce73d14bd7827b58b6bfbe5e7a => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 16:26:44.166189 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a9f8d3-52b5-4de1-aede-c988a19678b0 None None] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:26:44.166564 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a9f8d3-52b5-4de1-aede-c988a19678b0 None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:26:44.166789 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a9f8d3-52b5-4de1-aede-c988a19678b0 None None] PATH_INFO: `/` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:26:44.168974 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 74/149] 10.222.0.57 () {54 vars in 926 bytes} [Fri Jul 1 16:26:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 16:26:44.182278 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f0444f-8921-4ba3-a271-c8d2ec28a65e None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:26:44.182595 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f0444f-8921-4ba3-a271-c8d2ec28a65e None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:26:44.183585 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f0444f-8921-4ba3-a271-c8d2ec28a65e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:26:44.207615 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-91f0444f-8921-4ba3-a271-c8d2ec28a65e None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:26:44.208776 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-91f0444f-8921-4ba3-a271-c8d2ec28a65e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:26:44.228844 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:26:44.295804 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 76/150] 10.222.0.57 () {58 vars in 1023 bytes} [Fri Jul 1 16:26:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 01 16:26:44.337896 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a9f8d3-52b5-4de1-aede-c988a19678b0 None None] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:26:44.466987 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=100 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:26:44.472889 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a9f8d3-52b5-4de1-aede-c988a19678b0 None None] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:26:44.521531 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f6b112-b3b3-4029-bdb9-9abaaa362209 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:26:44.522378 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f6b112-b3b3-4029-bdb9-9abaaa362209 None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:26:44.522658 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f6b112-b3b3-4029-bdb9-9abaaa362209 None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:26:44.522945 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f6b112-b3b3-4029-bdb9-9abaaa362209 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:26:44.539721 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f6b112-b3b3-4029-bdb9-9abaaa362209 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 01 16:26:44.547648 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application [None req-e8f6b112-b3b3-4029-bdb9-9abaaa362209 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 01 16:26:44.547648 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 16:26:44.547648 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 16:26:44.547648 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 16:26:44.547648 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 16:26:44.547648 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 16:26:44.547648 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 16:26:44.547648 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 16:26:44.547648 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 16:26:44.547648 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 16:26:44.547648 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 16:26:44.547648 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 16:26:44.547648 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 01 16:26:44.547648 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 01 16:26:44.547648 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 01 16:26:44.547648 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 01 16:26:44.548517 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 01 16:26:44.548517 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 01 16:26:44.548517 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 01 16:26:44.548517 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 01 16:26:44.548517 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 16:26:44.548517 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 16:26:44.548517 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 16:26:44.548517 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 16:26:44.548517 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 16:26:44.548517 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 16:26:44.548517 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 16:26:44.548517 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application with Lock( Jul 01 16:26:44.548517 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 16:26:44.548517 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self._enter() Jul 01 16:26:44.548517 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 16:26:44.548517 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 16:26:44.549288 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 16:26:44.549288 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.creator() Jul 01 16:26:44.549288 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 16:26:44.549288 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application created_value = creator( Jul 01 16:26:44.549288 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 01 16:26:44.549288 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 01 16:26:44.549288 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 01 16:26:44.549288 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 01 16:26:44.549288 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 01 16:26:44.549288 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 01 16:26:44.549288 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 01 16:26:44.549288 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application Jul 01 16:26:44.549288 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 75/151] 10.222.0.57 () {56 vars in 1096 bytes} [Fri Jul 1 16:26:44 2022] GET /identity/v3/roles/admin => generated 83 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 16:26:44.554988 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f0444f-8921-4ba3-a271-c8d2ec28a65e None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:26:44.708938 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f0444f-8921-4ba3-a271-c8d2ec28a65e None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:26:44.758035 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85ab8bd-553c-447f-b44b-8c66d2a9e904 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:26:44.758662 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85ab8bd-553c-447f-b44b-8c66d2a9e904 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:26:44.758951 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85ab8bd-553c-447f-b44b-8c66d2a9e904 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:26:44.759229 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85ab8bd-553c-447f-b44b-8c66d2a9e904 None admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:26:44.760001 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f85ab8bd-553c-447f-b44b-8c66d2a9e904 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:26:44.760308 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f85ab8bd-553c-447f-b44b-8c66d2a9e904 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:26:44.762933 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f85ab8bd-553c-447f-b44b-8c66d2a9e904 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:26:44.776469 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 77/152] 10.222.0.57 () {56 vars in 1099 bytes} [Fri Jul 1 16:26:44 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:26:44.782802 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f6b112-b3b3-4029-bdb9-9abaaa362209 None admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:26:44.928644 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f6b112-b3b3-4029-bdb9-9abaaa362209 None admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:26:44.974749 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7cf02cc-9c32-40dc-8e8d-087b4cabf6c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:26:44.975640 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cf02cc-9c32-40dc-8e8d-087b4cabf6c1 None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:26:44.975924 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cf02cc-9c32-40dc-8e8d-087b4cabf6c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:26:44.976201 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cf02cc-9c32-40dc-8e8d-087b4cabf6c1 None admin] PATH_INFO: `/v3/users/9996f66c673f4673ab2342b66da9acc2` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:26:44.983111 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7cf02cc-9c32-40dc-8e8d-087b4cabf6c1 None admin] RBAC: Authorizing `identity:get_user(user_id=9996f66c673f4673ab2342b66da9acc2)` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:26:44.984638 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7cf02cc-9c32-40dc-8e8d-087b4cabf6c1 None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:26:44.986584 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 76/153] 10.222.0.57 () {56 vars in 1177 bytes} [Fri Jul 1 16:26:44 2022] GET /identity/v3/users/9996f66c673f4673ab2342b66da9acc2 => generated 247 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:26:44.992458 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85ab8bd-553c-447f-b44b-8c66d2a9e904 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:26:44.997729 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ef2852-55a3-4aab-a9e4-6881d9c69080 None None] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:26:44.998425 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ef2852-55a3-4aab-a9e4-6881d9c69080 None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:26:44.998930 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ef2852-55a3-4aab-a9e4-6881d9c69080 None None] PATH_INFO: `/` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:26:45.001635 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 77/154] 10.222.0.57 () {54 vars in 926 bytes} [Fri Jul 1 16:26:44 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 16:26:45.007788 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8ae442-f82a-4c0b-a90a-d8d555592b70 None None] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:26:45.008220 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8ae442-f82a-4c0b-a90a-d8d555592b70 None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:26:45.008377 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8ae442-f82a-4c0b-a90a-d8d555592b70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:26:45.029444 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:26:45.051840 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.auth.core [None req-eb8ae442-f82a-4c0b-a90a-d8d555592b70 None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80312) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:26:45.053596 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-eb8ae442-f82a-4c0b-a90a-d8d555592b70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:26:45.115836 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=100 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:26:45.201291 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85ab8bd-553c-447f-b44b-8c66d2a9e904 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:26:45.251915 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 78/155] 10.222.0.57 () {58 vars in 1023 bytes} [Fri Jul 1 16:26:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 245 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 01 16:26:45.267155 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f127f3a0-08d4-4896-924d-9464c0bc6371 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:26:45.267778 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f127f3a0-08d4-4896-924d-9464c0bc6371 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:26:45.268070 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f127f3a0-08d4-4896-924d-9464c0bc6371 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:26:45.268351 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f127f3a0-08d4-4896-924d-9464c0bc6371 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:26:45.284380 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8ae442-f82a-4c0b-a90a-d8d555592b70 None None] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:26:45.388401 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f127f3a0-08d4-4896-924d-9464c0bc6371 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:26:45.389885 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f127f3a0-08d4-4896-924d-9464c0bc6371 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:26:45.410575 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:26:45.554229 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8ae442-f82a-4c0b-a90a-d8d555592b70 None None] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:26:45.594830 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 78/156] 10.222.0.57 () {58 vars in 1282 bytes} [Fri Jul 1 16:26:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 603 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 01 16:26:45.600912 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f127f3a0-08d4-4896-924d-9464c0bc6371 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:26:45.618453 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67cbd354-f9b1-4219-aa87-e936c7295eef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:26:45.618453 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cbd354-f9b1-4219-aa87-e936c7295eef None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:26:45.618453 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cbd354-f9b1-4219-aa87-e936c7295eef None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:26:45.618453 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cbd354-f9b1-4219-aa87-e936c7295eef None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:26:45.625912 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67cbd354-f9b1-4219-aa87-e936c7295eef None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 01 16:26:45.638912 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application [None req-67cbd354-f9b1-4219-aa87-e936c7295eef None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 01 16:26:45.638912 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 16:26:45.638912 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 16:26:45.638912 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 16:26:45.638912 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 16:26:45.638912 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 16:26:45.638912 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 16:26:45.638912 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 16:26:45.638912 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 16:26:45.638912 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 16:26:45.638912 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 16:26:45.638912 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 16:26:45.638912 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 01 16:26:45.638912 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 01 16:26:45.638912 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 01 16:26:45.638912 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 01 16:26:45.640051 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 01 16:26:45.640051 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 01 16:26:45.640051 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 01 16:26:45.640051 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 01 16:26:45.640051 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 16:26:45.640051 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 16:26:45.640051 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 16:26:45.640051 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 16:26:45.640051 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 16:26:45.640051 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 16:26:45.640051 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 16:26:45.640051 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application with Lock( Jul 01 16:26:45.640051 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 16:26:45.640051 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self._enter() Jul 01 16:26:45.640051 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 16:26:45.640051 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 16:26:45.640921 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 16:26:45.640921 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.creator() Jul 01 16:26:45.640921 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 16:26:45.640921 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application created_value = creator( Jul 01 16:26:45.640921 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 01 16:26:45.640921 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 01 16:26:45.640921 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 01 16:26:45.640921 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 01 16:26:45.640921 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 01 16:26:45.640921 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 01 16:26:45.640921 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 01 16:26:45.640921 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application Jul 01 16:26:45.640921 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 79/157] 10.222.0.57 () {56 vars in 1114 bytes} [Fri Jul 1 16:26:45 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 356 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 16:26:45.641559 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5aafb0-efe0-4e38-ab18-d2742473d0c5 None None] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:26:45.641630 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5aafb0-efe0-4e38-ab18-d2742473d0c5 None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:26:45.642191 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5aafb0-efe0-4e38-ab18-d2742473d0c5 None None] PATH_INFO: `/` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:26:45.646377 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 80/158] 10.222.0.57 () {54 vars in 926 bytes} [Fri Jul 1 16:26:45 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 16:26:45.646377 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5aafb0-efe0-4e38-ab18-d2742473d0c5 None None] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:26:45.667254 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=100 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:26:45.756288 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5aafb0-efe0-4e38-ab18-d2742473d0c5 None None] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:26:45.794811 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a84ce8-41ea-4959-a2d3-35167a167e40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:26:45.795606 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a84ce8-41ea-4959-a2d3-35167a167e40 None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:26:45.795816 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a84ce8-41ea-4959-a2d3-35167a167e40 None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:26:45.796093 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a84ce8-41ea-4959-a2d3-35167a167e40 None admin] PATH_INFO: `/v3/roles` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:26:45.796908 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a84ce8-41ea-4959-a2d3-35167a167e40 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80312) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:26:45.797218 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a84ce8-41ea-4959-a2d3-35167a167e40 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:26:45.798838 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a84ce8-41ea-4959-a2d3-35167a167e40 None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:26:45.809153 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 81/159] 10.222.0.57 () {56 vars in 1111 bytes} [Fri Jul 1 16:26:45 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:26:45.810440 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43c691e-aba7-4183-a564-65ebb83280bc None None] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:26:45.810704 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43c691e-aba7-4183-a564-65ebb83280bc None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:26:45.810985 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43c691e-aba7-4183-a564-65ebb83280bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:26:45.829267 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.auth.core [None req-a43c691e-aba7-4183-a564-65ebb83280bc None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80312) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:26:45.830529 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-a43c691e-aba7-4183-a564-65ebb83280bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:26:45.853894 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f127f3a0-08d4-4896-924d-9464c0bc6371 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:26:45.879916 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:26:45.903859 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 82/160] 10.222.0.57 () {58 vars in 1023 bytes} [Fri Jul 1 16:26:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 01 16:26:45.904726 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43c691e-aba7-4183-a564-65ebb83280bc None None] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:26:45.922764 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=100 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:26:45.987076 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450c5eb2-96a0-4392-ac7d-21afc0c75680 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:26:45.988431 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450c5eb2-96a0-4392-ac7d-21afc0c75680 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:26:45.988923 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450c5eb2-96a0-4392-ac7d-21afc0c75680 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:26:45.989408 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450c5eb2-96a0-4392-ac7d-21afc0c75680 None admin] PATH_INFO: `/v3/projects/5bef204363a04d5d8ceab527b8803f6e` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:26:45.990950 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-450c5eb2-96a0-4392-ac7d-21afc0c75680 None admin] RBAC: Authorizing `identity:get_project(project_id=5bef204363a04d5d8ceab527b8803f6e)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:26:45.992623 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-450c5eb2-96a0-4392-ac7d-21afc0c75680 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:26:45.994625 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 79/161] 10.222.0.57 () {56 vars in 1186 bytes} [Fri Jul 1 16:26:45 2022] GET /identity/v3/projects/5bef204363a04d5d8ceab527b8803f6e => generated 298 bytes in 394 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 01 16:26:45.995750 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450c5eb2-96a0-4392-ac7d-21afc0c75680 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:26:46.036402 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43c691e-aba7-4183-a564-65ebb83280bc None None] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:26:46.074915 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9abe0a-8653-45db-b251-a2c564d148b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:26:46.076136 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9abe0a-8653-45db-b251-a2c564d148b8 None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:26:46.076574 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9abe0a-8653-45db-b251-a2c564d148b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:26:46.077014 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9abe0a-8653-45db-b251-a2c564d148b8 None admin] PATH_INFO: `/v3/users/dcca9b98066044dca1d7d028b9e83c21` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:26:46.078500 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf9abe0a-8653-45db-b251-a2c564d148b8 None admin] RBAC: Authorizing `identity:get_user(user_id=dcca9b98066044dca1d7d028b9e83c21)` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:26:46.080081 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf9abe0a-8653-45db-b251-a2c564d148b8 None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:26:46.082216 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 83/162] 10.222.0.57 () {56 vars in 1177 bytes} [Fri Jul 1 16:26:45 2022] GET /identity/v3/users/dcca9b98066044dca1d7d028b9e83c21 => generated 275 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:26:46.083415 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9abe0a-8653-45db-b251-a2c564d148b8 None admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:26:46.166731 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=100 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:26:46.194656 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9abe0a-8653-45db-b251-a2c564d148b8 None admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:26:46.222959 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450c5eb2-96a0-4392-ac7d-21afc0c75680 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:26:46.248322 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255433f7-77a2-44c5-a215-b2482e598724 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:26:46.249314 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255433f7-77a2-44c5-a215-b2482e598724 None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:26:46.249738 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255433f7-77a2-44c5-a215-b2482e598724 None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:26:46.250182 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255433f7-77a2-44c5-a215-b2482e598724 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:26:46.252884 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-255433f7-77a2-44c5-a215-b2482e598724 None admin] RBAC: Adding query filter params (role.id=84ee739146934c1ab32c501ff8093644, scope.project.id=5bef204363a04d5d8ceab527b8803f6e, user.id=9996f66c673f4673ab2342b66da9acc2) {{(pid=80312) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:26:46.253444 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-255433f7-77a2-44c5-a215-b2482e598724 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:26:46.255130 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-255433f7-77a2-44c5-a215-b2482e598724 None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:26:46.267108 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 84/163] 10.222.0.57 () {56 vars in 1374 bytes} [Fri Jul 1 16:26:46 2022] GET /identity/v3/role_assignments?user.id=9996f66c673f4673ab2342b66da9acc2&scope.project.id=5bef204363a04d5d8ceab527b8803f6e&role.id=84ee739146934c1ab32c501ff8093644 => generated 608 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:26:46.268407 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255433f7-77a2-44c5-a215-b2482e598724 None admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:26:46.295793 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa9c125-d59f-458e-8175-13b6d3afd9e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:26:46.296611 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa9c125-d59f-458e-8175-13b6d3afd9e9 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:26:46.296908 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa9c125-d59f-458e-8175-13b6d3afd9e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:26:46.297454 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa9c125-d59f-458e-8175-13b6d3afd9e9 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:26:46.324422 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa9c125-d59f-458e-8175-13b6d3afd9e9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 01 16:26:46.340958 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application [None req-bfa9c125-d59f-458e-8175-13b6d3afd9e9 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 01 16:26:46.340958 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 16:26:46.340958 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 16:26:46.340958 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 16:26:46.340958 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 16:26:46.340958 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 16:26:46.340958 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 16:26:46.340958 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 16:26:46.340958 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 16:26:46.340958 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 16:26:46.340958 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 16:26:46.340958 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 16:26:46.340958 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 01 16:26:46.340958 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 01 16:26:46.340958 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 01 16:26:46.340958 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 01 16:26:46.341876 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 01 16:26:46.341876 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 01 16:26:46.341876 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 01 16:26:46.341876 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 01 16:26:46.341876 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 16:26:46.341876 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 16:26:46.341876 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 16:26:46.341876 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 16:26:46.341876 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 16:26:46.341876 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 16:26:46.341876 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 16:26:46.341876 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application with Lock( Jul 01 16:26:46.341876 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 16:26:46.341876 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._enter() Jul 01 16:26:46.341876 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 16:26:46.341876 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 16:26:46.342700 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 16:26:46.342700 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.creator() Jul 01 16:26:46.342700 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 16:26:46.342700 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application created_value = creator( Jul 01 16:26:46.342700 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 01 16:26:46.342700 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 01 16:26:46.342700 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 01 16:26:46.342700 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 01 16:26:46.342700 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 01 16:26:46.342700 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 01 16:26:46.342700 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 01 16:26:46.342700 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Jul 01 16:26:46.342700 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 80/164] 10.222.0.57 () {56 vars in 1099 bytes} [Fri Jul 1 16:26:45 2022] GET /identity/v3/roles/member => generated 84 bytes in 347 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 16:26:46.351259 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa9c125-d59f-458e-8175-13b6d3afd9e9 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:26:46.406883 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255433f7-77a2-44c5-a215-b2482e598724 None admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:26:46.432392 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:26:46.448629 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f0a41e-de00-4e16-9e4d-992efdc0c06d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:26:46.449773 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f0a41e-de00-4e16-9e4d-992efdc0c06d None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:26:46.450251 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f0a41e-de00-4e16-9e4d-992efdc0c06d None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:26:46.450742 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f0a41e-de00-4e16-9e4d-992efdc0c06d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:26:46.486745 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=100 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:26:46.528560 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f0a41e-de00-4e16-9e4d-992efdc0c06d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:26:46.530530 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f0a41e-de00-4e16-9e4d-992efdc0c06d None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:26:46.534080 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa9c125-d59f-458e-8175-13b6d3afd9e9 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:26:46.611073 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2415939-4ea7-49ed-bc8d-7344b8e58db6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:26:46.611682 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2415939-4ea7-49ed-bc8d-7344b8e58db6 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:26:46.611972 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2415939-4ea7-49ed-bc8d-7344b8e58db6 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:26:46.612260 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2415939-4ea7-49ed-bc8d-7344b8e58db6 None admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:26:46.613042 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2415939-4ea7-49ed-bc8d-7344b8e58db6 None admin] RBAC: Adding query filter params (name=member) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:26:46.613359 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2415939-4ea7-49ed-bc8d-7344b8e58db6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:26:46.614999 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2415939-4ea7-49ed-bc8d-7344b8e58db6 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:26:46.624399 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 81/165] 10.222.0.57 () {56 vars in 1101 bytes} [Fri Jul 1 16:26:46 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:26:46.639162 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2415939-4ea7-49ed-bc8d-7344b8e58db6 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:26:46.672593 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 85/166] 10.222.0.57 () {58 vars in 1282 bytes} [Fri Jul 1 16:26:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 405 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 01 16:26:46.678373 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f0a41e-de00-4e16-9e4d-992efdc0c06d None admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:26:46.728452 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=100 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:26:46.792724 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f0a41e-de00-4e16-9e4d-992efdc0c06d None admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:26:46.814685 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:26:46.820726 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2415939-4ea7-49ed-bc8d-7344b8e58db6 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:26:46.833700 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8030114-e0cf-48e5-bd2d-5bc95a3ad997 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:26:46.835449 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8030114-e0cf-48e5-bd2d-5bc95a3ad997 None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:26:46.836040 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8030114-e0cf-48e5-bd2d-5bc95a3ad997 None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:26:46.836512 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8030114-e0cf-48e5-bd2d-5bc95a3ad997 None admin] PATH_INFO: `/v3/projects/5bef204363a04d5d8ceab527b8803f6e` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:26:46.838058 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8030114-e0cf-48e5-bd2d-5bc95a3ad997 None admin] RBAC: Authorizing `identity:get_project(project_id=5bef204363a04d5d8ceab527b8803f6e)` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:26:46.839845 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8030114-e0cf-48e5-bd2d-5bc95a3ad997 None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:26:46.841954 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 86/167] 10.222.0.57 () {56 vars in 1186 bytes} [Fri Jul 1 16:26:46 2022] GET /identity/v3/projects/5bef204363a04d5d8ceab527b8803f6e => generated 298 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 01 16:26:46.859423 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8030114-e0cf-48e5-bd2d-5bc95a3ad997 None admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:26:46.893560 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a967672-c122-4f1c-9e53-b45598e63d61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:26:46.894395 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a967672-c122-4f1c-9e53-b45598e63d61 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:26:46.894747 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a967672-c122-4f1c-9e53-b45598e63d61 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:26:46.895035 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a967672-c122-4f1c-9e53-b45598e63d61 None admin] PATH_INFO: `/v3/users/dcca9b98066044dca1d7d028b9e83c21` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:26:46.896282 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a967672-c122-4f1c-9e53-b45598e63d61 None admin] RBAC: Authorizing `identity:get_user(user_id=dcca9b98066044dca1d7d028b9e83c21)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:26:46.897763 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a967672-c122-4f1c-9e53-b45598e63d61 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:26:46.899758 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 82/168] 10.222.0.57 () {56 vars in 1177 bytes} [Fri Jul 1 16:26:46 2022] GET /identity/v3/users/dcca9b98066044dca1d7d028b9e83c21 => generated 275 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:26:46.907326 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a967672-c122-4f1c-9e53-b45598e63d61 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:26:47.014820 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8030114-e0cf-48e5-bd2d-5bc95a3ad997 None admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:26:47.028655 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=100 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:26:47.054584 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a967672-c122-4f1c-9e53-b45598e63d61 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:26:47.061340 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0a0ec7-51ed-4d73-81ef-f632ff4ccf01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:26:47.062009 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0a0ec7-51ed-4d73-81ef-f632ff4ccf01 None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:26:47.062291 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0a0ec7-51ed-4d73-81ef-f632ff4ccf01 None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:26:47.065039 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0a0ec7-51ed-4d73-81ef-f632ff4ccf01 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:26:47.066099 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e0a0ec7-51ed-4d73-81ef-f632ff4ccf01 None admin] RBAC: Adding query filter params (role.id=a8e5c73da81d4f0eb7b92c7058e500fe, scope.project.id=5bef204363a04d5d8ceab527b8803f6e, user.id=dcca9b98066044dca1d7d028b9e83c21) {{(pid=80312) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:26:47.066614 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e0a0ec7-51ed-4d73-81ef-f632ff4ccf01 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:26:47.068279 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e0a0ec7-51ed-4d73-81ef-f632ff4ccf01 None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:26:47.075968 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 87/169] 10.222.0.57 () {56 vars in 1374 bytes} [Fri Jul 1 16:26:46 2022] GET /identity/v3/role_assignments?user.id=dcca9b98066044dca1d7d028b9e83c21&scope.project.id=5bef204363a04d5d8ceab527b8803f6e&role.id=a8e5c73da81d4f0eb7b92c7058e500fe => generated 608 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:26:47.113858 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b8ac5d-a5ff-4199-a757-72a571e50d0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:26:47.114559 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b8ac5d-a5ff-4199-a757-72a571e50d0c None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:26:47.114849 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b8ac5d-a5ff-4199-a757-72a571e50d0c None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:26:47.115132 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b8ac5d-a5ff-4199-a757-72a571e50d0c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:26:47.161819 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:26:47.208964 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b8ac5d-a5ff-4199-a757-72a571e50d0c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:26:47.210684 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b8ac5d-a5ff-4199-a757-72a571e50d0c None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:26:47.335398 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 83/170] 10.222.0.57 () {58 vars in 1282 bytes} [Fri Jul 1 16:26:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 428 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 01 16:26:47.341708 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0a0ec7-51ed-4d73-81ef-f632ff4ccf01 None admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:26:47.491272 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0a0ec7-51ed-4d73-81ef-f632ff4ccf01 None admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:26:47.533867 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce691ff-6d97-484e-9340-cf928a44e90d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:26:47.534794 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce691ff-6d97-484e-9340-cf928a44e90d None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:26:47.535081 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce691ff-6d97-484e-9340-cf928a44e90d None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:26:47.535360 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce691ff-6d97-484e-9340-cf928a44e90d None admin] PATH_INFO: `/v3/projects/bf1bfeff7565472da796cac2f99ad6bb` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:26:47.536608 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce691ff-6d97-484e-9340-cf928a44e90d None admin] RBAC: Authorizing `identity:get_project(project_id=bf1bfeff7565472da796cac2f99ad6bb)` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:26:47.538062 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce691ff-6d97-484e-9340-cf928a44e90d None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:26:47.540016 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 88/171] 10.222.0.57 () {56 vars in 1186 bytes} [Fri Jul 1 16:26:47 2022] GET /identity/v3/projects/bf1bfeff7565472da796cac2f99ad6bb => generated 312 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:26:47.546679 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b8ac5d-a5ff-4199-a757-72a571e50d0c None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:26:47.603865 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ba2ef8-ce10-4935-a610-7595715ced3f None None] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:26:47.604166 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ba2ef8-ce10-4935-a610-7595715ced3f None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:26:47.605169 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ba2ef8-ce10-4935-a610-7595715ced3f None None] PATH_INFO: `/` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:26:47.607106 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 89/172] 10.222.0.57 () {54 vars in 926 bytes} [Fri Jul 1 16:26:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 01 16:26:47.613666 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c71adf7-f9c9-4884-bf1a-cdb9926f717e None None] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:26:47.613958 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c71adf7-f9c9-4884-bf1a-cdb9926f717e None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:26:47.614244 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c71adf7-f9c9-4884-bf1a-cdb9926f717e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:26:47.643448 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.auth.core [None req-8c71adf7-f9c9-4884-bf1a-cdb9926f717e None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80312) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:26:47.644586 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-8c71adf7-f9c9-4884-bf1a-cdb9926f717e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:26:47.649192 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:26:47.693602 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b8ac5d-a5ff-4199-a757-72a571e50d0c None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:26:47.733590 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=100 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:26:47.748669 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58df4222-ee12-4ead-8cd8-7450b7f3a4ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:26:47.749653 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58df4222-ee12-4ead-8cd8-7450b7f3a4ed None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:26:47.750147 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58df4222-ee12-4ead-8cd8-7450b7f3a4ed None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:26:47.750659 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58df4222-ee12-4ead-8cd8-7450b7f3a4ed None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:26:47.751845 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58df4222-ee12-4ead-8cd8-7450b7f3a4ed None admin] RBAC: Adding query filter params (role.id=983be40bdab04c478eaa90c6b6a46b14, scope.project.id=bf1bfeff7565472da796cac2f99ad6bb, user.id=dcca9b98066044dca1d7d028b9e83c21) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:26:47.752355 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58df4222-ee12-4ead-8cd8-7450b7f3a4ed None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:26:47.754152 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58df4222-ee12-4ead-8cd8-7450b7f3a4ed None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:26:47.768348 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 84/173] 10.222.0.57 () {56 vars in 1374 bytes} [Fri Jul 1 16:26:47 2022] GET /identity/v3/role_assignments?user.id=dcca9b98066044dca1d7d028b9e83c21&scope.project.id=bf1bfeff7565472da796cac2f99ad6bb&role.id=983be40bdab04c478eaa90c6b6a46b14 => generated 608 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:26:47.816070 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 90/174] 10.222.0.57 () {58 vars in 1023 bytes} [Fri Jul 1 16:26:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Jul 01 16:26:47.820575 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708e1584-0dc3-448d-9731-0ef5e7d11a4b None None] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:26:47.821101 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708e1584-0dc3-448d-9731-0ef5e7d11a4b None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:26:47.821575 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708e1584-0dc3-448d-9731-0ef5e7d11a4b None None] PATH_INFO: `/` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:26:47.823235 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 85/175] 10.222.0.57 () {54 vars in 926 bytes} [Fri Jul 1 16:26:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 16:26:47.829301 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f02a698-4dba-49e0-a69b-9446058d9f37 None None] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:26:47.829591 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f02a698-4dba-49e0-a69b-9446058d9f37 None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:26:47.829876 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f02a698-4dba-49e0-a69b-9446058d9f37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:26:47.865666 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708e1584-0dc3-448d-9731-0ef5e7d11a4b None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:26:47.881748 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.auth.core [None req-1f02a698-4dba-49e0-a69b-9446058d9f37 None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80312) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:26:47.881748 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-1f02a698-4dba-49e0-a69b-9446058d9f37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:26:48.021375 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 91/176] 10.222.0.57 () {58 vars in 1023 bytes} [Fri Jul 1 16:26:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 01 16:26:48.023054 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c066bb5c-c749-4f05-ab7d-4c62220708c1 None None] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:26:48.023543 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c066bb5c-c749-4f05-ab7d-4c62220708c1 None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:26:48.024077 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c066bb5c-c749-4f05-ab7d-4c62220708c1 None None] PATH_INFO: `/` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:26:48.025683 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 92/177] 10.222.0.57 () {54 vars in 926 bytes} [Fri Jul 1 16:26:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 16:26:48.053909 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fa5ba1-9099-4afe-bae5-0b96c77b496c None None] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:26:48.054543 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fa5ba1-9099-4afe-bae5-0b96c77b496c None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:26:48.055032 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fa5ba1-9099-4afe-bae5-0b96c77b496c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:26:48.062598 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708e1584-0dc3-448d-9731-0ef5e7d11a4b None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:26:48.073879 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.auth.core [None req-99fa5ba1-9099-4afe-bae5-0b96c77b496c None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80312) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:26:48.075480 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-99fa5ba1-9099-4afe-bae5-0b96c77b496c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:26:48.104047 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:26:48.113786 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f136589-2ada-46a0-bc9e-14a5a2357bdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:26:48.114647 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f136589-2ada-46a0-bc9e-14a5a2357bdc None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:26:48.115483 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f136589-2ada-46a0-bc9e-14a5a2357bdc None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:26:48.115483 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f136589-2ada-46a0-bc9e-14a5a2357bdc None admin] PATH_INFO: `/v3/roles/member` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:26:48.129668 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f136589-2ada-46a0-bc9e-14a5a2357bdc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 01 16:26:48.137477 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application [None req-4f136589-2ada-46a0-bc9e-14a5a2357bdc None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 01 16:26:48.137477 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 16:26:48.137477 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 16:26:48.137477 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 16:26:48.137477 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 16:26:48.137477 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 16:26:48.137477 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 16:26:48.137477 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 16:26:48.137477 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 16:26:48.137477 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 16:26:48.137477 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 16:26:48.137477 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 16:26:48.137477 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 01 16:26:48.137477 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 01 16:26:48.137477 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 01 16:26:48.137477 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 01 16:26:48.138498 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 01 16:26:48.138498 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 01 16:26:48.138498 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 01 16:26:48.138498 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 01 16:26:48.138498 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 16:26:48.138498 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 16:26:48.138498 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 16:26:48.138498 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 16:26:48.138498 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 16:26:48.138498 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 16:26:48.138498 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 16:26:48.138498 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application with Lock( Jul 01 16:26:48.138498 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 16:26:48.138498 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._enter() Jul 01 16:26:48.138498 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 16:26:48.138498 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 16:26:48.139301 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 16:26:48.139301 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.creator() Jul 01 16:26:48.139301 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 16:26:48.139301 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application created_value = creator( Jul 01 16:26:48.139301 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 01 16:26:48.139301 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 01 16:26:48.139301 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 01 16:26:48.139301 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 01 16:26:48.139301 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 01 16:26:48.139301 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 01 16:26:48.139301 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 01 16:26:48.139301 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Jul 01 16:26:48.139904 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 86/178] 10.222.0.57 () {56 vars in 1099 bytes} [Fri Jul 1 16:26:47 2022] GET /identity/v3/roles/member => generated 84 bytes in 275 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 16:26:48.140594 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f136589-2ada-46a0-bc9e-14a5a2357bdc None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:26:48.180952 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 93/179] 10.222.0.57 () {58 vars in 1023 bytes} [Fri Jul 1 16:26:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 01 16:26:48.183019 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1ae5c0-99f6-4318-8deb-b71625540dfd None None] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:26:48.183449 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1ae5c0-99f6-4318-8deb-b71625540dfd None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:26:48.183871 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1ae5c0-99f6-4318-8deb-b71625540dfd None None] PATH_INFO: `/` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:26:48.186003 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 94/180] 10.222.0.57 () {54 vars in 926 bytes} [Fri Jul 1 16:26:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 16:26:48.192686 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1ae5c0-99f6-4318-8deb-b71625540dfd None None] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:26:48.230217 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=100 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:26:48.281124 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f136589-2ada-46a0-bc9e-14a5a2357bdc None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:26:48.310748 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1ae5c0-99f6-4318-8deb-b71625540dfd None None] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:26:48.319756 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556dde22-100a-4531-a718-67c9722fedd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:26:48.320918 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556dde22-100a-4531-a718-67c9722fedd9 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:26:48.321208 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556dde22-100a-4531-a718-67c9722fedd9 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:26:48.321777 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556dde22-100a-4531-a718-67c9722fedd9 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:26:48.336371 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556dde22-100a-4531-a718-67c9722fedd9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 01 16:26:48.347427 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application [None req-556dde22-100a-4531-a718-67c9722fedd9 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 01 16:26:48.347427 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 16:26:48.347427 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 16:26:48.347427 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 16:26:48.347427 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 16:26:48.347427 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 16:26:48.347427 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 16:26:48.347427 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 16:26:48.347427 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 16:26:48.347427 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 16:26:48.347427 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 16:26:48.347427 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 16:26:48.347427 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 01 16:26:48.347427 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 01 16:26:48.347427 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 01 16:26:48.347427 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 01 16:26:48.349049 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 01 16:26:48.349049 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 01 16:26:48.349049 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 01 16:26:48.349049 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 01 16:26:48.349049 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 16:26:48.349049 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 16:26:48.349049 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 16:26:48.349049 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 16:26:48.349049 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 16:26:48.349049 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 16:26:48.349049 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 16:26:48.349049 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application with Lock( Jul 01 16:26:48.349049 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 16:26:48.349049 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._enter() Jul 01 16:26:48.349049 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 16:26:48.349049 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 16:26:48.351638 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 16:26:48.351638 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.creator() Jul 01 16:26:48.351638 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 16:26:48.351638 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application created_value = creator( Jul 01 16:26:48.351638 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 01 16:26:48.351638 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 01 16:26:48.351638 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 01 16:26:48.351638 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 01 16:26:48.351638 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 01 16:26:48.351638 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 01 16:26:48.351638 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 01 16:26:48.351638 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Jul 01 16:26:48.353468 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656fce4b-8419-46cd-ac9e-d6ecd18c974a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:26:48.353468 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656fce4b-8419-46cd-ac9e-d6ecd18c974a None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:26:48.353468 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656fce4b-8419-46cd-ac9e-d6ecd18c974a None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:26:48.353468 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656fce4b-8419-46cd-ac9e-d6ecd18c974a None admin] PATH_INFO: `/v3/roles` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:26:48.355122 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 87/181] 10.222.0.57 () {56 vars in 1114 bytes} [Fri Jul 1 16:26:48 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 16:26:48.355269 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-656fce4b-8419-46cd-ac9e-d6ecd18c974a None admin] RBAC: Adding query filter params (name=member) {{(pid=80312) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:26:48.355269 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-656fce4b-8419-46cd-ac9e-d6ecd18c974a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:26:48.355537 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27c382b-bde9-4ada-a4ab-c66b6b520914 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:26:48.355678 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-656fce4b-8419-46cd-ac9e-d6ecd18c974a None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:26:48.355813 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27c382b-bde9-4ada-a4ab-c66b6b520914 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:26:48.355813 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27c382b-bde9-4ada-a4ab-c66b6b520914 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:26:48.360371 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 95/182] 10.222.0.57 () {56 vars in 1101 bytes} [Fri Jul 1 16:26:48 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:26:48.361177 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656fce4b-8419-46cd-ac9e-d6ecd18c974a None admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:26:48.372808 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-a27c382b-bde9-4ada-a4ab-c66b6b520914 None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:26:48.374015 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-a27c382b-bde9-4ada-a4ab-c66b6b520914 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:26:48.421055 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:26:48.441008 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 88/183] 10.222.0.57 () {58 vars in 1023 bytes} [Fri Jul 1 16:26:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 01 16:26:48.442339 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af36dfb5-2ed3-4ae7-98ab-9add7a5e7cd5 None None] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:26:48.442656 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af36dfb5-2ed3-4ae7-98ab-9add7a5e7cd5 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:26:48.442934 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af36dfb5-2ed3-4ae7-98ab-9add7a5e7cd5 None None] PATH_INFO: `/` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:26:48.444353 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 89/184] 10.222.0.57 () {54 vars in 926 bytes} [Fri Jul 1 16:26:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 16:26:48.444907 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af36dfb5-2ed3-4ae7-98ab-9add7a5e7cd5 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:26:48.465177 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=100 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:26:48.475205 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656fce4b-8419-46cd-ac9e-d6ecd18c974a None admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:26:48.509506 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8e3cf2-d3a8-424c-9858-f73ac2394b22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:26:48.510310 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8e3cf2-d3a8-424c-9858-f73ac2394b22 None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:26:48.511151 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8e3cf2-d3a8-424c-9858-f73ac2394b22 None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:26:48.511434 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8e3cf2-d3a8-424c-9858-f73ac2394b22 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:26:48.521752 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e8e3cf2-d3a8-424c-9858-f73ac2394b22 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 01 16:26:48.529655 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application [None req-2e8e3cf2-d3a8-424c-9858-f73ac2394b22 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 01 16:26:48.529655 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 16:26:48.529655 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 16:26:48.529655 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 16:26:48.529655 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 16:26:48.529655 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 16:26:48.529655 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 16:26:48.529655 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 16:26:48.529655 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 16:26:48.529655 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 16:26:48.529655 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 16:26:48.529655 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 16:26:48.529655 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 01 16:26:48.529655 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 01 16:26:48.529655 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 01 16:26:48.529655 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 01 16:26:48.530599 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 01 16:26:48.530599 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 01 16:26:48.530599 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 01 16:26:48.530599 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 01 16:26:48.530599 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 16:26:48.530599 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 16:26:48.530599 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 16:26:48.530599 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 16:26:48.530599 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 16:26:48.530599 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 16:26:48.530599 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 16:26:48.530599 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application with Lock( Jul 01 16:26:48.530599 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 16:26:48.530599 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self._enter() Jul 01 16:26:48.530599 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 16:26:48.530599 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 16:26:48.531419 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 16:26:48.531419 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.creator() Jul 01 16:26:48.531419 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 16:26:48.531419 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application created_value = creator( Jul 01 16:26:48.531419 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 01 16:26:48.531419 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 01 16:26:48.531419 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 01 16:26:48.531419 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 01 16:26:48.531419 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 01 16:26:48.531419 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 01 16:26:48.531419 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 01 16:26:48.531419 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application Jul 01 16:26:48.531419 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 96/185] 10.222.0.57 () {56 vars in 1096 bytes} [Fri Jul 1 16:26:48 2022] GET /identity/v3/roles/admin => generated 83 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 16:26:48.532079 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8e3cf2-d3a8-424c-9858-f73ac2394b22 None admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:26:48.561059 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af36dfb5-2ed3-4ae7-98ab-9add7a5e7cd5 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:26:48.594808 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376f5a1b-f4dc-4c82-b25c-db691e10ac2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:26:48.595472 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376f5a1b-f4dc-4c82-b25c-db691e10ac2c None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:26:48.595753 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376f5a1b-f4dc-4c82-b25c-db691e10ac2c None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:26:48.596033 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376f5a1b-f4dc-4c82-b25c-db691e10ac2c None admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:26:48.596777 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-376f5a1b-f4dc-4c82-b25c-db691e10ac2c None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:26:48.597085 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-376f5a1b-f4dc-4c82-b25c-db691e10ac2c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:26:48.598504 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-376f5a1b-f4dc-4c82-b25c-db691e10ac2c None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:26:48.606932 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 90/186] 10.222.0.57 () {56 vars in 1111 bytes} [Fri Jul 1 16:26:48 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:26:48.608132 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a29aec5-be07-40d8-b1bc-cd77e385ca23 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:26:48.608431 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a29aec5-be07-40d8-b1bc-cd77e385ca23 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:26:48.608698 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a29aec5-be07-40d8-b1bc-cd77e385ca23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:26:48.625401 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-7a29aec5-be07-40d8-b1bc-cd77e385ca23 None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:26:48.626649 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-7a29aec5-be07-40d8-b1bc-cd77e385ca23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:26:48.636987 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8e3cf2-d3a8-424c-9858-f73ac2394b22 None admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:26:48.666123 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:26:48.671278 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a518dc74-fd3e-4c06-8720-677a3adc20ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:26:48.672143 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a518dc74-fd3e-4c06-8720-677a3adc20ea None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:26:48.672431 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a518dc74-fd3e-4c06-8720-677a3adc20ea None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:26:48.672708 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a518dc74-fd3e-4c06-8720-677a3adc20ea None admin] PATH_INFO: `/v3/users/dcca9b98066044dca1d7d028b9e83c21` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:26:48.673905 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a518dc74-fd3e-4c06-8720-677a3adc20ea None admin] RBAC: Authorizing `identity:get_user(user_id=dcca9b98066044dca1d7d028b9e83c21)` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:26:48.675400 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a518dc74-fd3e-4c06-8720-677a3adc20ea None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:26:48.677034 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 97/187] 10.222.0.57 () {56 vars in 1177 bytes} [Fri Jul 1 16:26:48 2022] GET /identity/v3/users/dcca9b98066044dca1d7d028b9e83c21 => generated 275 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:26:48.677802 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a518dc74-fd3e-4c06-8720-677a3adc20ea None admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:26:48.697125 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 91/188] 10.222.0.57 () {58 vars in 1023 bytes} [Fri Jul 1 16:26:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 01 16:26:48.697975 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a29aec5-be07-40d8-b1bc-cd77e385ca23 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:26:48.702945 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=100 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:26:48.783463 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a518dc74-fd3e-4c06-8720-677a3adc20ea None admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:26:48.796790 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a29aec5-be07-40d8-b1bc-cd77e385ca23 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:26:48.816527 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5bb4ec8-5220-4086-be7a-96fb6cbeb419 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:26:48.817325 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bb4ec8-5220-4086-be7a-96fb6cbeb419 None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:26:48.817608 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bb4ec8-5220-4086-be7a-96fb6cbeb419 None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:26:48.817882 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bb4ec8-5220-4086-be7a-96fb6cbeb419 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:26:48.827899 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5bb4ec8-5220-4086-be7a-96fb6cbeb419 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 01 16:26:48.829883 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d236b0f3-a510-45c4-9a8d-56af23109b65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:26:48.830850 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d236b0f3-a510-45c4-9a8d-56af23109b65 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:26:48.831146 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d236b0f3-a510-45c4-9a8d-56af23109b65 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:26:48.831419 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d236b0f3-a510-45c4-9a8d-56af23109b65 None admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:26:48.832167 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d236b0f3-a510-45c4-9a8d-56af23109b65 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:26:48.832471 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d236b0f3-a510-45c4-9a8d-56af23109b65 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:26:48.833837 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d236b0f3-a510-45c4-9a8d-56af23109b65 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:26:48.836000 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application [None req-d5bb4ec8-5220-4086-be7a-96fb6cbeb419 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 01 16:26:48.836000 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 16:26:48.836000 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 16:26:48.836000 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 16:26:48.836000 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 16:26:48.836000 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 16:26:48.836000 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 16:26:48.836000 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 16:26:48.836000 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 16:26:48.836000 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 16:26:48.836000 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 16:26:48.836000 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 16:26:48.836000 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 01 16:26:48.836000 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 01 16:26:48.836000 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 01 16:26:48.836000 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 01 16:26:48.837048 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 01 16:26:48.837048 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 01 16:26:48.837048 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 01 16:26:48.837048 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 01 16:26:48.837048 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 16:26:48.837048 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 16:26:48.837048 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 16:26:48.837048 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 16:26:48.837048 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 16:26:48.837048 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 16:26:48.837048 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 16:26:48.837048 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application with Lock( Jul 01 16:26:48.837048 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 16:26:48.837048 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self._enter() Jul 01 16:26:48.837048 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 16:26:48.837048 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 16:26:48.838094 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 16:26:48.838094 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.creator() Jul 01 16:26:48.838094 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 16:26:48.838094 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application created_value = creator( Jul 01 16:26:48.838094 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 01 16:26:48.838094 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 01 16:26:48.838094 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 01 16:26:48.838094 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 01 16:26:48.838094 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 01 16:26:48.838094 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 01 16:26:48.838094 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 01 16:26:48.838094 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application Jul 01 16:26:48.838094 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 98/189] 10.222.0.57 () {56 vars in 1099 bytes} [Fri Jul 1 16:26:48 2022] GET /identity/v3/roles/reader => generated 84 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 16:26:48.843405 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5bb4ec8-5220-4086-be7a-96fb6cbeb419 None admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:26:48.846108 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 92/190] 10.222.0.57 () {56 vars in 1099 bytes} [Fri Jul 1 16:26:48 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:26:48.846811 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d236b0f3-a510-45c4-9a8d-56af23109b65 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:26:48.883904 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:26:48.925294 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=100 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:26:48.944376 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5bb4ec8-5220-4086-be7a-96fb6cbeb419 None admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:26:48.950961 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d236b0f3-a510-45c4-9a8d-56af23109b65 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:26:48.977866 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb2fc78-1e20-4eef-85fd-d3a926a76c43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:26:48.978779 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb2fc78-1e20-4eef-85fd-d3a926a76c43 None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:26:48.979061 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb2fc78-1e20-4eef-85fd-d3a926a76c43 None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:26:48.979409 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb2fc78-1e20-4eef-85fd-d3a926a76c43 None admin] PATH_INFO: `/v3/users/6b9b08aa7935439db0311fdcd6ba14f8` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:26:48.987101 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdbef468-093a-4709-af19-576e991a94bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:26:48.987861 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbef468-093a-4709-af19-576e991a94bc None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:26:48.988466 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbef468-093a-4709-af19-576e991a94bc None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:26:48.988466 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbef468-093a-4709-af19-576e991a94bc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:26:48.993075 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb2fc78-1e20-4eef-85fd-d3a926a76c43 None admin] RBAC: Authorizing `identity:get_user(user_id=6b9b08aa7935439db0311fdcd6ba14f8)` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:26:48.994577 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb2fc78-1e20-4eef-85fd-d3a926a76c43 None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:26:48.997224 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 99/191] 10.222.0.57 () {56 vars in 1177 bytes} [Fri Jul 1 16:26:48 2022] GET /identity/v3/users/6b9b08aa7935439db0311fdcd6ba14f8 => generated 283 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:26:48.997839 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb2fc78-1e20-4eef-85fd-d3a926a76c43 None admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:26:49.061642 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdbef468-093a-4709-af19-576e991a94bc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:26:49.063038 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdbef468-093a-4709-af19-576e991a94bc None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:26:49.095129 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb2fc78-1e20-4eef-85fd-d3a926a76c43 None admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:26:49.096329 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:26:49.129693 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c20463d-cfb5-4cce-b737-a43e77702649 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:26:49.130608 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c20463d-cfb5-4cce-b737-a43e77702649 None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:26:49.130961 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c20463d-cfb5-4cce-b737-a43e77702649 None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:26:49.131243 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c20463d-cfb5-4cce-b737-a43e77702649 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:26:49.132402 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c20463d-cfb5-4cce-b737-a43e77702649 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:26:49.133831 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c20463d-cfb5-4cce-b737-a43e77702649 None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:26:49.135668 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 100/192] 10.222.0.57 () {56 vars in 1108 bytes} [Fri Jul 1 16:26:48 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:26:49.136464 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c20463d-cfb5-4cce-b737-a43e77702649 None admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:26:49.144872 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=100 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:26:49.167824 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 93/193] 10.222.0.57 () {58 vars in 1282 bytes} [Fri Jul 1 16:26:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 01 16:26:49.168470 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdbef468-093a-4709-af19-576e991a94bc None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:26:49.240188 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c20463d-cfb5-4cce-b737-a43e77702649 None admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:26:49.270547 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdbef468-093a-4709-af19-576e991a94bc None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:26:49.275655 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67707d03-6af8-4aa6-a6b9-bbb2b6634639 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:26:49.276886 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67707d03-6af8-4aa6-a6b9-bbb2b6634639 None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:26:49.277177 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67707d03-6af8-4aa6-a6b9-bbb2b6634639 None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:26:49.277455 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67707d03-6af8-4aa6-a6b9-bbb2b6634639 None admin] PATH_INFO: `/v3/roles` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:26:49.278209 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67707d03-6af8-4aa6-a6b9-bbb2b6634639 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80312) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:26:49.278547 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67707d03-6af8-4aa6-a6b9-bbb2b6634639 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:26:49.279942 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67707d03-6af8-4aa6-a6b9-bbb2b6634639 None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:26:49.287916 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 101/194] 10.222.0.57 () {56 vars in 1101 bytes} [Fri Jul 1 16:26:49 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:26:49.289651 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67707d03-6af8-4aa6-a6b9-bbb2b6634639 None admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:26:49.308359 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da24fd7-55be-4529-a6fa-5cb19b029721 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:26:49.309236 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da24fd7-55be-4529-a6fa-5cb19b029721 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:26:49.309524 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da24fd7-55be-4529-a6fa-5cb19b029721 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:26:49.309806 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da24fd7-55be-4529-a6fa-5cb19b029721 None admin] PATH_INFO: `/v3/users/6b9b08aa7935439db0311fdcd6ba14f8` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:26:49.311066 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da24fd7-55be-4529-a6fa-5cb19b029721 None admin] RBAC: Authorizing `identity:get_user(user_id=6b9b08aa7935439db0311fdcd6ba14f8)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:26:49.312560 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da24fd7-55be-4529-a6fa-5cb19b029721 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:26:49.314150 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 94/195] 10.222.0.57 () {56 vars in 1177 bytes} [Fri Jul 1 16:26:49 2022] GET /identity/v3/users/6b9b08aa7935439db0311fdcd6ba14f8 => generated 283 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:26:49.314925 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da24fd7-55be-4529-a6fa-5cb19b029721 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:26:49.320481 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:26:49.363836 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=100 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:26:49.398507 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67707d03-6af8-4aa6-a6b9-bbb2b6634639 None admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:26:49.420827 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da24fd7-55be-4529-a6fa-5cb19b029721 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:26:49.433927 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa65767-7481-4c30-aef3-6443fe570666 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:26:49.435009 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa65767-7481-4c30-aef3-6443fe570666 None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:26:49.435443 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa65767-7481-4c30-aef3-6443fe570666 None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:26:49.435734 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa65767-7481-4c30-aef3-6443fe570666 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:26:49.453306 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7e00f3-6266-405a-ab44-5e7028a8e65a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:26:49.453983 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7e00f3-6266-405a-ab44-5e7028a8e65a None admin] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:26:49.454267 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7e00f3-6266-405a-ab44-5e7028a8e65a None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:26:49.454568 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7e00f3-6266-405a-ab44-5e7028a8e65a None admin] PATH_INFO: `/v3/users` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:26:49.455272 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd7e00f3-6266-405a-ab44-5e7028a8e65a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:26:49.456972 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd7e00f3-6266-405a-ab44-5e7028a8e65a None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:26:49.483714 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 95/196] 10.222.0.57 () {60 vars in 1132 bytes} [Fri Jul 1 16:26:49 2022] POST /identity/v3/users => generated 297 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 16:26:49.484882 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7e00f3-6266-405a-ab44-5e7028a8e65a None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:26:49.527950 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa65767-7481-4c30-aef3-6443fe570666 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:26:49.529599 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa65767-7481-4c30-aef3-6443fe570666 None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:26:49.556612 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:26:49.615686 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7e00f3-6266-405a-ab44-5e7028a8e65a None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:26:49.618049 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=100 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:26:49.660569 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 102/197] 10.222.0.57 () {58 vars in 1282 bytes} [Fri Jul 1 16:26:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 01 16:26:49.661436 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa65767-7481-4c30-aef3-6443fe570666 None admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:26:49.666389 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0d4ae4-f5a1-49bc-8fe9-fcfd3627857f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:26:49.667451 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0d4ae4-f5a1-49bc-8fe9-fcfd3627857f None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:26:49.667737 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0d4ae4-f5a1-49bc-8fe9-fcfd3627857f None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:26:49.667998 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0d4ae4-f5a1-49bc-8fe9-fcfd3627857f None admin] PATH_INFO: `/v3/projects/5bef204363a04d5d8ceab527b8803f6e` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:26:49.669226 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd0d4ae4-f5a1-49bc-8fe9-fcfd3627857f None admin] RBAC: Authorizing `identity:get_project(project_id=5bef204363a04d5d8ceab527b8803f6e)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:26:49.670786 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd0d4ae4-f5a1-49bc-8fe9-fcfd3627857f None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:26:49.672494 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 96/198] 10.222.0.57 () {56 vars in 1186 bytes} [Fri Jul 1 16:26:49 2022] GET /identity/v3/projects/5bef204363a04d5d8ceab527b8803f6e => generated 298 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:26:49.678954 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0d4ae4-f5a1-49bc-8fe9-fcfd3627857f None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:26:49.800781 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa65767-7481-4c30-aef3-6443fe570666 None admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:26:49.811831 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0d4ae4-f5a1-49bc-8fe9-fcfd3627857f None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:26:49.846733 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2559ec88-2d7f-4a68-b40c-b69e61dd57dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:26:49.848219 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:26:49.848400 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2559ec88-2d7f-4a68-b40c-b69e61dd57dd None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:26:49.848665 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2559ec88-2d7f-4a68-b40c-b69e61dd57dd None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:26:49.849126 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2559ec88-2d7f-4a68-b40c-b69e61dd57dd None admin] PATH_INFO: `/v3/users/3d26974a32f240fd853a2dee432b4586` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:26:49.851557 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2559ec88-2d7f-4a68-b40c-b69e61dd57dd None admin] RBAC: Authorizing `identity:get_user(user_id=3d26974a32f240fd853a2dee432b4586)` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:26:49.853498 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2559ec88-2d7f-4a68-b40c-b69e61dd57dd None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:26:49.856673 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 103/199] 10.222.0.57 () {56 vars in 1177 bytes} [Fri Jul 1 16:26:49 2022] GET /identity/v3/users/3d26974a32f240fd853a2dee432b4586 => generated 289 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:26:49.861115 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2559ec88-2d7f-4a68-b40c-b69e61dd57dd None admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:26:49.863498 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da80a655-0d0e-44cf-a38d-213695c9bc11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:26:49.864348 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da80a655-0d0e-44cf-a38d-213695c9bc11 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:26:49.864477 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da80a655-0d0e-44cf-a38d-213695c9bc11 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:26:49.864729 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da80a655-0d0e-44cf-a38d-213695c9bc11 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:26:49.914492 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=100 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:26:49.942881 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da80a655-0d0e-44cf-a38d-213695c9bc11 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:26:49.944405 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da80a655-0d0e-44cf-a38d-213695c9bc11 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:26:49.976447 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2559ec88-2d7f-4a68-b40c-b69e61dd57dd None admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:26:50.012875 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24ebf4e-dcf2-41bf-b349-51666c5cae29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:26:50.013757 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24ebf4e-dcf2-41bf-b349-51666c5cae29 None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:26:50.014052 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24ebf4e-dcf2-41bf-b349-51666c5cae29 None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:26:50.014357 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24ebf4e-dcf2-41bf-b349-51666c5cae29 None admin] PATH_INFO: `/v3/projects/63f84f2aa0bf4707bd9551422f0cae3e` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:26:50.015840 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a24ebf4e-dcf2-41bf-b349-51666c5cae29 None admin] RBAC: Authorizing `identity:get_project(project_id=63f84f2aa0bf4707bd9551422f0cae3e)` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:26:50.017303 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a24ebf4e-dcf2-41bf-b349-51666c5cae29 None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:26:50.019315 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 104/200] 10.222.0.57 () {56 vars in 1186 bytes} [Fri Jul 1 16:26:49 2022] GET /identity/v3/projects/63f84f2aa0bf4707bd9551422f0cae3e => generated 302 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:26:50.024748 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24ebf4e-dcf2-41bf-b349-51666c5cae29 None admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:26:50.063850 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 97/201] 10.222.0.57 () {58 vars in 1282 bytes} [Fri Jul 1 16:26:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 385 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 01 16:26:50.064551 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da80a655-0d0e-44cf-a38d-213695c9bc11 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:26:50.099205 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:26:50.130371 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24ebf4e-dcf2-41bf-b349-51666c5cae29 None admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:26:50.138723 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=100 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:26:50.170989 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfdb2a7a-431b-49c4-95b4-772bf36852f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:26:50.171617 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdb2a7a-431b-49c4-95b4-772bf36852f0 None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:26:50.171909 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdb2a7a-431b-49c4-95b4-772bf36852f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:26:50.172199 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdb2a7a-431b-49c4-95b4-772bf36852f0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:26:50.173054 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfdb2a7a-431b-49c4-95b4-772bf36852f0 None admin] RBAC: Adding query filter params (role.id=983be40bdab04c478eaa90c6b6a46b14, scope.project.id=5bef204363a04d5d8ceab527b8803f6e, user.id=dcca9b98066044dca1d7d028b9e83c21) {{(pid=80312) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:26:50.173375 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfdb2a7a-431b-49c4-95b4-772bf36852f0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:26:50.174881 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfdb2a7a-431b-49c4-95b4-772bf36852f0 None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:26:50.181174 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 105/202] 10.222.0.57 () {56 vars in 1374 bytes} [Fri Jul 1 16:26:50 2022] GET /identity/v3/role_assignments?user.id=dcca9b98066044dca1d7d028b9e83c21&scope.project.id=5bef204363a04d5d8ceab527b8803f6e&role.id=983be40bdab04c478eaa90c6b6a46b14 => generated 608 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:26:50.182097 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfdb2a7a-431b-49c4-95b4-772bf36852f0 None admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:26:50.204802 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da80a655-0d0e-44cf-a38d-213695c9bc11 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:26:50.277051 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc133713-af88-47ff-8aff-e2c7de170044 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:26:50.277726 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc133713-af88-47ff-8aff-e2c7de170044 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:26:50.278015 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc133713-af88-47ff-8aff-e2c7de170044 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:26:50.278298 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc133713-af88-47ff-8aff-e2c7de170044 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:26:50.294250 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfdb2a7a-431b-49c4-95b4-772bf36852f0 None admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:26:50.332501 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e70168-5771-4e42-98e5-990bff198f3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:26:50.333176 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e70168-5771-4e42-98e5-990bff198f3a None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:26:50.333469 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e70168-5771-4e42-98e5-990bff198f3a None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:26:50.333756 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e70168-5771-4e42-98e5-990bff198f3a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:26:50.334634 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e70168-5771-4e42-98e5-990bff198f3a None admin] RBAC: Adding query filter params (role.id=a8e5c73da81d4f0eb7b92c7058e500fe, scope.project.id=63f84f2aa0bf4707bd9551422f0cae3e, user.id=6b9b08aa7935439db0311fdcd6ba14f8) {{(pid=80312) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:26:50.334960 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e70168-5771-4e42-98e5-990bff198f3a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:26:50.336449 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e70168-5771-4e42-98e5-990bff198f3a None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:26:50.343185 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 106/203] 10.222.0.57 () {56 vars in 1374 bytes} [Fri Jul 1 16:26:50 2022] GET /identity/v3/role_assignments?user.id=6b9b08aa7935439db0311fdcd6ba14f8&scope.project.id=63f84f2aa0bf4707bd9551422f0cae3e&role.id=a8e5c73da81d4f0eb7b92c7058e500fe => generated 259 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:26:50.347257 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e70168-5771-4e42-98e5-990bff198f3a None admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:26:50.441223 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=100 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:26:50.454223 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc133713-af88-47ff-8aff-e2c7de170044 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:26:50.456027 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc133713-af88-47ff-8aff-e2c7de170044 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:26:50.500830 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:26:50.570135 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e70168-5771-4e42-98e5-990bff198f3a None admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:26:50.592464 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 98/204] 10.222.0.57 () {58 vars in 1282 bytes} [Fri Jul 1 16:26:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 528 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 01 16:26:50.598670 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc133713-af88-47ff-8aff-e2c7de170044 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:26:50.609219 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb63c2e7-0c89-4550-a66b-401f246dbba3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:26:50.610276 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb63c2e7-0c89-4550-a66b-401f246dbba3 None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:26:50.610673 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb63c2e7-0c89-4550-a66b-401f246dbba3 None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:26:50.610963 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb63c2e7-0c89-4550-a66b-401f246dbba3 None admin] PATH_INFO: `/v3/projects/63f84f2aa0bf4707bd9551422f0cae3e` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:26:50.612193 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb63c2e7-0c89-4550-a66b-401f246dbba3 None admin] RBAC: Authorizing `identity:get_project(project_id=63f84f2aa0bf4707bd9551422f0cae3e)` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:26:50.613645 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb63c2e7-0c89-4550-a66b-401f246dbba3 None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:26:50.615733 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 107/205] 10.222.0.57 () {56 vars in 1186 bytes} [Fri Jul 1 16:26:50 2022] GET /identity/v3/projects/63f84f2aa0bf4707bd9551422f0cae3e => generated 302 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:26:50.622473 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb63c2e7-0c89-4550-a66b-401f246dbba3 None admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:26:50.737992 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=100 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:26:50.741274 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb63c2e7-0c89-4550-a66b-401f246dbba3 None admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:26:50.783068 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2afa81d7-698b-4a6f-b11c-595fb61ba605 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:26:50.783743 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afa81d7-698b-4a6f-b11c-595fb61ba605 None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:26:50.784035 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afa81d7-698b-4a6f-b11c-595fb61ba605 None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:26:50.784321 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afa81d7-698b-4a6f-b11c-595fb61ba605 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:26:50.785166 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2afa81d7-698b-4a6f-b11c-595fb61ba605 None admin] RBAC: Adding query filter params (role.id=84ee739146934c1ab32c501ff8093644, scope.project.id=63f84f2aa0bf4707bd9551422f0cae3e, user.id=6b9b08aa7935439db0311fdcd6ba14f8) {{(pid=80312) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:26:50.785486 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2afa81d7-698b-4a6f-b11c-595fb61ba605 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:26:50.788875 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2afa81d7-698b-4a6f-b11c-595fb61ba605 None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:26:50.793679 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc133713-af88-47ff-8aff-e2c7de170044 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:26:50.804787 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 108/206] 10.222.0.57 () {56 vars in 1374 bytes} [Fri Jul 1 16:26:50 2022] GET /identity/v3/role_assignments?user.id=6b9b08aa7935439db0311fdcd6ba14f8&scope.project.id=63f84f2aa0bf4707bd9551422f0cae3e&role.id=84ee739146934c1ab32c501ff8093644 => generated 259 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:26:50.846774 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:26:50.863619 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b486da9f-e0c6-42e8-b137-ec79de877bf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:26:50.865092 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b486da9f-e0c6-42e8-b137-ec79de877bf5 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:26:50.865574 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b486da9f-e0c6-42e8-b137-ec79de877bf5 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:26:50.866070 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b486da9f-e0c6-42e8-b137-ec79de877bf5 None admin] PATH_INFO: `/v3/projects/5bef204363a04d5d8ceab527b8803f6e` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:26:50.867624 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b486da9f-e0c6-42e8-b137-ec79de877bf5 None admin] RBAC: Authorizing `identity:get_project(project_id=5bef204363a04d5d8ceab527b8803f6e)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:26:50.869151 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b486da9f-e0c6-42e8-b137-ec79de877bf5 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:26:50.871326 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 99/207] 10.222.0.57 () {56 vars in 1186 bytes} [Fri Jul 1 16:26:50 2022] GET /identity/v3/projects/5bef204363a04d5d8ceab527b8803f6e => generated 298 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:26:50.877532 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2afa81d7-698b-4a6f-b11c-595fb61ba605 None admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:26:51.009960 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2afa81d7-698b-4a6f-b11c-595fb61ba605 None admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:26:51.051191 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd4508e-e5f9-4544-87d3-c6d45286c717 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:26:51.051863 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd4508e-e5f9-4544-87d3-c6d45286c717 None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:26:51.052151 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd4508e-e5f9-4544-87d3-c6d45286c717 None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:26:51.052440 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd4508e-e5f9-4544-87d3-c6d45286c717 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:26:51.053287 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd4508e-e5f9-4544-87d3-c6d45286c717 None admin] RBAC: Adding query filter params (role.id=bd7cefce73d14bd7827b58b6bfbe5e7a, scope.project.id=5bef204363a04d5d8ceab527b8803f6e, user.id=3d26974a32f240fd853a2dee432b4586) {{(pid=80312) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:26:51.053604 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd4508e-e5f9-4544-87d3-c6d45286c717 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:26:51.055091 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd4508e-e5f9-4544-87d3-c6d45286c717 None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:26:51.061319 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 109/208] 10.222.0.57 () {56 vars in 1374 bytes} [Fri Jul 1 16:26:50 2022] GET /identity/v3/role_assignments?user.id=3d26974a32f240fd853a2dee432b4586&scope.project.id=5bef204363a04d5d8ceab527b8803f6e&role.id=bd7cefce73d14bd7827b58b6bfbe5e7a => generated 608 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:26:51.980800 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776bcc46-917e-4159-a12c-771bf5aa93c7 None None] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:26:51.981304 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776bcc46-917e-4159-a12c-771bf5aa93c7 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:26:51.981377 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776bcc46-917e-4159-a12c-771bf5aa93c7 None None] PATH_INFO: `/` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:26:51.983073 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 100/209] 10.222.0.57 () {54 vars in 926 bytes} [Fri Jul 1 16:26:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 16:26:51.988341 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a33297-5871-4857-adfd-af7c1bd58e7e None None] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:26:51.988641 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a33297-5871-4857-adfd-af7c1bd58e7e None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:26:51.988928 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a33297-5871-4857-adfd-af7c1bd58e7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:26:52.005957 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.auth.core [None req-59a33297-5871-4857-adfd-af7c1bd58e7e None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80312) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:26:52.007189 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-59a33297-5871-4857-adfd-af7c1bd58e7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:26:52.032626 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=100 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:26:52.082450 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 110/210] 10.222.0.57 () {58 vars in 1023 bytes} [Fri Jul 1 16:26:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 01 16:26:52.106390 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56668ff3-3b2b-4941-b5ac-aaa58c535fa3 None None] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:26:52.106630 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56668ff3-3b2b-4941-b5ac-aaa58c535fa3 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:26:52.106865 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56668ff3-3b2b-4941-b5ac-aaa58c535fa3 None None] PATH_INFO: `/` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:26:52.108407 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 101/211] 10.222.0.57 () {54 vars in 926 bytes} [Fri Jul 1 16:26:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 16:26:52.112281 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a33297-5871-4857-adfd-af7c1bd58e7e None None] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:26:52.113491 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4a4fe8-dfff-4cd4-a0b2-5f848f38c587 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:26:52.113788 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4a4fe8-dfff-4cd4-a0b2-5f848f38c587 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:26:52.114078 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4a4fe8-dfff-4cd4-a0b2-5f848f38c587 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:26:52.138508 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-3e4a4fe8-dfff-4cd4-a0b2-5f848f38c587 None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:26:52.139994 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-3e4a4fe8-dfff-4cd4-a0b2-5f848f38c587 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:26:52.228399 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a33297-5871-4857-adfd-af7c1bd58e7e None None] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:26:52.232512 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 102/212] 10.222.0.57 () {58 vars in 1023 bytes} [Fri Jul 1 16:26:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 01 16:26:52.261567 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4a4fe8-dfff-4cd4-a0b2-5f848f38c587 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:26:52.268549 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c23b5c3-08c0-4f5e-ad6b-fda6c7ab0eab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:26:52.269620 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c23b5c3-08c0-4f5e-ad6b-fda6c7ab0eab None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:26:52.270109 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c23b5c3-08c0-4f5e-ad6b-fda6c7ab0eab None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:26:52.270620 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c23b5c3-08c0-4f5e-ad6b-fda6c7ab0eab None admin] PATH_INFO: `/v3/domains/default` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:26:52.272001 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c23b5c3-08c0-4f5e-ad6b-fda6c7ab0eab None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:26:52.273720 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c23b5c3-08c0-4f5e-ad6b-fda6c7ab0eab None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:26:52.275628 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 111/213] 10.222.0.57 () {56 vars in 1108 bytes} [Fri Jul 1 16:26:52 2022] GET /identity/v3/domains/default => generated 199 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:26:52.281866 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c23b5c3-08c0-4f5e-ad6b-fda6c7ab0eab None admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:26:52.311133 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=100 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:26:52.373662 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4a4fe8-dfff-4cd4-a0b2-5f848f38c587 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:26:52.389798 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c23b5c3-08c0-4f5e-ad6b-fda6c7ab0eab None admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:26:52.402555 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:26:52.412048 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053b4c87-bb12-406c-824d-9cb477dfcb55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:26:52.412849 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053b4c87-bb12-406c-824d-9cb477dfcb55 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:26:52.413134 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053b4c87-bb12-406c-824d-9cb477dfcb55 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:26:52.413413 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053b4c87-bb12-406c-824d-9cb477dfcb55 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:26:52.424924 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-053b4c87-bb12-406c-824d-9cb477dfcb55 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 01 16:26:52.430645 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9f845e-c132-4aee-980a-de9432f6228c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:26:52.431640 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9f845e-c132-4aee-980a-de9432f6228c None admin] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:26:52.432681 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9f845e-c132-4aee-980a-de9432f6228c None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:26:52.433226 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9f845e-c132-4aee-980a-de9432f6228c None admin] PATH_INFO: `/v3/users` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:26:52.433721 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application [None req-053b4c87-bb12-406c-824d-9cb477dfcb55 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 01 16:26:52.433721 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 16:26:52.433721 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 16:26:52.433721 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 16:26:52.433721 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 16:26:52.433721 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 16:26:52.433721 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 16:26:52.433721 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 16:26:52.433721 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 16:26:52.433721 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 16:26:52.433721 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 16:26:52.433721 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 16:26:52.433721 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 01 16:26:52.433721 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 01 16:26:52.433721 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 01 16:26:52.433721 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 01 16:26:52.434627 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 01 16:26:52.434627 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 01 16:26:52.434627 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 01 16:26:52.434627 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 01 16:26:52.434627 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 16:26:52.434627 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 16:26:52.434627 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 16:26:52.434627 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 16:26:52.434627 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 16:26:52.434627 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 16:26:52.434627 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 16:26:52.434627 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application with Lock( Jul 01 16:26:52.434627 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 16:26:52.434627 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._enter() Jul 01 16:26:52.434627 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 16:26:52.434627 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 16:26:52.435429 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 16:26:52.435429 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.creator() Jul 01 16:26:52.435429 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 16:26:52.435429 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application created_value = creator( Jul 01 16:26:52.435429 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 01 16:26:52.435429 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 01 16:26:52.435429 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 01 16:26:52.435429 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 01 16:26:52.435429 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 01 16:26:52.435429 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 01 16:26:52.435429 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 01 16:26:52.435429 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Jul 01 16:26:52.437004 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9f845e-c132-4aee-980a-de9432f6228c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:26:52.438779 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9f845e-c132-4aee-980a-de9432f6228c None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:26:52.439839 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 103/214] 10.222.0.57 () {56 vars in 1099 bytes} [Fri Jul 1 16:26:52 2022] GET /identity/v3/roles/member => generated 84 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 16:26:52.441109 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053b4c87-bb12-406c-824d-9cb477dfcb55 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:26:52.464649 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 112/215] 10.222.0.57 () {60 vars in 1132 bytes} [Fri Jul 1 16:26:52 2022] POST /identity/v3/users => generated 297 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 16:26:52.570453 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053b4c87-bb12-406c-824d-9cb477dfcb55 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:26:52.612532 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2f6e61-c4b1-4808-933f-bd814d47b68e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:26:52.613633 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2f6e61-c4b1-4808-933f-bd814d47b68e None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:26:52.614143 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2f6e61-c4b1-4808-933f-bd814d47b68e None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:26:52.614643 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2f6e61-c4b1-4808-933f-bd814d47b68e None admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:26:52.616183 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e2f6e61-c4b1-4808-933f-bd814d47b68e None admin] RBAC: Adding query filter params (name=member) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:26:52.616830 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e2f6e61-c4b1-4808-933f-bd814d47b68e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:26:52.618454 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e2f6e61-c4b1-4808-933f-bd814d47b68e None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:26:52.628184 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 104/216] 10.222.0.57 () {56 vars in 1101 bytes} [Fri Jul 1 16:26:52 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:26:52.634063 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9f845e-c132-4aee-980a-de9432f6228c None admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:26:52.707116 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74e47de-0ca0-4428-ac29-9cb19e1678de None None] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:26:52.707436 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74e47de-0ca0-4428-ac29-9cb19e1678de None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:26:52.707774 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74e47de-0ca0-4428-ac29-9cb19e1678de None None] PATH_INFO: `/` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:26:52.709188 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 105/217] 10.222.0.57 () {54 vars in 926 bytes} [Fri Jul 1 16:26:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 16:26:52.715510 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1136f6d-b18f-4b56-8eb9-5a091d1d3714 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:26:52.716082 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1136f6d-b18f-4b56-8eb9-5a091d1d3714 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:26:52.716571 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1136f6d-b18f-4b56-8eb9-5a091d1d3714 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:26:52.728663 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:26:52.742194 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-c1136f6d-b18f-4b56-8eb9-5a091d1d3714 None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:26:52.743419 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-c1136f6d-b18f-4b56-8eb9-5a091d1d3714 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:26:52.753398 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9f845e-c132-4aee-980a-de9432f6228c None admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:26:52.802885 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f077a200-3a87-4c70-a90b-bfecf9f78e74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:26:52.803396 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:26:52.804840 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f077a200-3a87-4c70-a90b-bfecf9f78e74 None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:26:52.805229 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f077a200-3a87-4c70-a90b-bfecf9f78e74 None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:26:52.805515 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f077a200-3a87-4c70-a90b-bfecf9f78e74 None admin] PATH_INFO: `/v3/users/04bead9a70014c4e8222b6d571279eb3` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:26:52.825797 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f077a200-3a87-4c70-a90b-bfecf9f78e74 None admin] RBAC: Authorizing `identity:get_user(user_id=04bead9a70014c4e8222b6d571279eb3)` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:26:52.827505 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f077a200-3a87-4c70-a90b-bfecf9f78e74 None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:26:52.831200 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 113/218] 10.222.0.57 () {56 vars in 1177 bytes} [Fri Jul 1 16:26:52 2022] GET /identity/v3/users/04bead9a70014c4e8222b6d571279eb3 => generated 297 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 01 16:26:52.838530 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f077a200-3a87-4c70-a90b-bfecf9f78e74 None admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:26:52.845331 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 106/219] 10.222.0.57 () {58 vars in 1023 bytes} [Fri Jul 1 16:26:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 01 16:26:52.869220 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1136f6d-b18f-4b56-8eb9-5a091d1d3714 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:26:52.969070 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f077a200-3a87-4c70-a90b-bfecf9f78e74 None admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:26:53.007001 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1136f6d-b18f-4b56-8eb9-5a091d1d3714 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:26:53.011429 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1dce01-4fcd-4055-9868-1fd5a98559b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:26:53.012133 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1dce01-4fcd-4055-9868-1fd5a98559b7 None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:26:53.012414 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1dce01-4fcd-4055-9868-1fd5a98559b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:26:53.012736 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1dce01-4fcd-4055-9868-1fd5a98559b7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:26:53.032598 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=100 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:26:53.051457 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40626a7-8d35-4623-9ed3-7729aa8ba33a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:26:53.052725 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40626a7-8d35-4623-9ed3-7729aa8ba33a None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:26:53.053311 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40626a7-8d35-4623-9ed3-7729aa8ba33a None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:26:53.053808 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40626a7-8d35-4623-9ed3-7729aa8ba33a None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:26:53.065795 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d40626a7-8d35-4623-9ed3-7729aa8ba33a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 01 16:26:53.075383 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application [None req-d40626a7-8d35-4623-9ed3-7729aa8ba33a None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 01 16:26:53.075383 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 16:26:53.075383 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 16:26:53.075383 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 16:26:53.075383 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 16:26:53.075383 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 16:26:53.075383 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 16:26:53.075383 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 16:26:53.075383 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 16:26:53.075383 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 16:26:53.075383 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 16:26:53.075383 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 16:26:53.075383 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 01 16:26:53.075383 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 01 16:26:53.075383 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 01 16:26:53.075383 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 01 16:26:53.076280 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 01 16:26:53.076280 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 01 16:26:53.076280 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 01 16:26:53.076280 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 01 16:26:53.076280 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 16:26:53.076280 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 16:26:53.076280 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 16:26:53.076280 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 16:26:53.076280 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 16:26:53.076280 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 16:26:53.076280 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 16:26:53.076280 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application with Lock( Jul 01 16:26:53.076280 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 16:26:53.076280 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._enter() Jul 01 16:26:53.076280 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 16:26:53.076280 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 16:26:53.077084 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 16:26:53.077084 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.creator() Jul 01 16:26:53.077084 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 16:26:53.077084 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application created_value = creator( Jul 01 16:26:53.077084 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 01 16:26:53.077084 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 01 16:26:53.077084 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 01 16:26:53.077084 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 01 16:26:53.077084 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 01 16:26:53.077084 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 01 16:26:53.077084 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 01 16:26:53.077084 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Jul 01 16:26:53.079699 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 107/220] 10.222.0.57 () {56 vars in 1114 bytes} [Fri Jul 1 16:26:52 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 01 16:26:53.081649 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed21b5b-35e5-48c3-bf9e-61913012a159 None None] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:26:53.082141 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed21b5b-35e5-48c3-bf9e-61913012a159 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:26:53.082716 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed21b5b-35e5-48c3-bf9e-61913012a159 None None] PATH_INFO: `/` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:26:53.084203 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 108/221] 10.222.0.57 () {54 vars in 926 bytes} [Fri Jul 1 16:26:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 16:26:53.089250 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed21b5b-35e5-48c3-bf9e-61913012a159 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:26:53.092280 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b1dce01-4fcd-4055-9868-1fd5a98559b7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:26:53.094485 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b1dce01-4fcd-4055-9868-1fd5a98559b7 None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:26:53.138500 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:26:53.201403 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed21b5b-35e5-48c3-bf9e-61913012a159 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:26:53.204775 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 114/222] 10.222.0.57 () {58 vars in 1282 bytes} [Fri Jul 1 16:26:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 01 16:26:53.206314 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41cb78f9-dc01-47e7-ab71-89d8cc6d38a2 None None] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:26:53.211044 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41cb78f9-dc01-47e7-ab71-89d8cc6d38a2 None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:26:53.211341 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41cb78f9-dc01-47e7-ab71-89d8cc6d38a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:26:53.229753 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.auth.core [None req-41cb78f9-dc01-47e7-ab71-89d8cc6d38a2 None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80312) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:26:53.231080 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-41cb78f9-dc01-47e7-ab71-89d8cc6d38a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:26:53.241206 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45c90d3-1fa0-450f-b4fb-e3bec7bfd4a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:26:53.242095 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45c90d3-1fa0-450f-b4fb-e3bec7bfd4a6 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:26:53.242551 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45c90d3-1fa0-450f-b4fb-e3bec7bfd4a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:26:53.242998 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45c90d3-1fa0-450f-b4fb-e3bec7bfd4a6 None admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:26:53.243903 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a45c90d3-1fa0-450f-b4fb-e3bec7bfd4a6 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:26:53.244358 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a45c90d3-1fa0-450f-b4fb-e3bec7bfd4a6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:26:53.246523 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a45c90d3-1fa0-450f-b4fb-e3bec7bfd4a6 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:26:53.255312 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 109/223] 10.222.0.57 () {56 vars in 1111 bytes} [Fri Jul 1 16:26:53 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:26:53.256489 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45c90d3-1fa0-450f-b4fb-e3bec7bfd4a6 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:26:53.269830 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=100 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:26:53.314369 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 115/224] 10.222.0.57 () {58 vars in 1023 bytes} [Fri Jul 1 16:26:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 01 16:26:53.314369 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41cb78f9-dc01-47e7-ab71-89d8cc6d38a2 None None] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:26:53.384308 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:26:53.387690 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45c90d3-1fa0-450f-b4fb-e3bec7bfd4a6 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:26:53.423462 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014e6994-3ed9-45fe-8fd8-6547ffddc9ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:26:53.424672 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014e6994-3ed9-45fe-8fd8-6547ffddc9ed None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:26:53.425159 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014e6994-3ed9-45fe-8fd8-6547ffddc9ed None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:26:53.425636 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014e6994-3ed9-45fe-8fd8-6547ffddc9ed None admin] PATH_INFO: `/v3/projects/63f84f2aa0bf4707bd9551422f0cae3e` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:26:53.427107 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-014e6994-3ed9-45fe-8fd8-6547ffddc9ed None admin] RBAC: Authorizing `identity:get_project(project_id=63f84f2aa0bf4707bd9551422f0cae3e)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:26:53.428857 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-014e6994-3ed9-45fe-8fd8-6547ffddc9ed None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:26:53.431156 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 110/225] 10.222.0.57 () {56 vars in 1186 bytes} [Fri Jul 1 16:26:53 2022] GET /identity/v3/projects/63f84f2aa0bf4707bd9551422f0cae3e => generated 302 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:26:53.432761 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014e6994-3ed9-45fe-8fd8-6547ffddc9ed None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:26:53.438675 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41cb78f9-dc01-47e7-ab71-89d8cc6d38a2 None None] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:26:53.479350 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8238a7-2100-4f89-bd72-1e2e615ea2be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:26:53.480390 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8238a7-2100-4f89-bd72-1e2e615ea2be None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:26:53.480620 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8238a7-2100-4f89-bd72-1e2e615ea2be None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:26:53.480896 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8238a7-2100-4f89-bd72-1e2e615ea2be None admin] PATH_INFO: `/v3/users/6b9b08aa7935439db0311fdcd6ba14f8` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:26:53.482103 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec8238a7-2100-4f89-bd72-1e2e615ea2be None admin] RBAC: Authorizing `identity:get_user(user_id=6b9b08aa7935439db0311fdcd6ba14f8)` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:26:53.483682 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec8238a7-2100-4f89-bd72-1e2e615ea2be None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:26:53.485823 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 116/226] 10.222.0.57 () {56 vars in 1177 bytes} [Fri Jul 1 16:26:53 2022] GET /identity/v3/users/6b9b08aa7935439db0311fdcd6ba14f8 => generated 283 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:26:53.486731 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8238a7-2100-4f89-bd72-1e2e615ea2be None admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:26:53.532931 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=100 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:26:53.547037 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014e6994-3ed9-45fe-8fd8-6547ffddc9ed None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:26:53.582916 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2c293f-38fc-4d43-bd73-e7b26aa816f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:26:53.584080 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2c293f-38fc-4d43-bd73-e7b26aa816f5 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:26:53.584570 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2c293f-38fc-4d43-bd73-e7b26aa816f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:26:53.585064 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2c293f-38fc-4d43-bd73-e7b26aa816f5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:26:53.596613 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2c293f-38fc-4d43-bd73-e7b26aa816f5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 01 16:26:53.598298 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8238a7-2100-4f89-bd72-1e2e615ea2be None admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:26:53.605575 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application [None req-0e2c293f-38fc-4d43-bd73-e7b26aa816f5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 01 16:26:53.605575 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 16:26:53.605575 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 16:26:53.605575 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 16:26:53.605575 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 16:26:53.605575 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 16:26:53.605575 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 16:26:53.605575 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 16:26:53.605575 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 16:26:53.605575 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 16:26:53.605575 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 16:26:53.605575 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 16:26:53.605575 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 01 16:26:53.605575 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 01 16:26:53.605575 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 01 16:26:53.605575 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 01 16:26:53.606714 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 01 16:26:53.606714 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 01 16:26:53.606714 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 01 16:26:53.606714 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 01 16:26:53.606714 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 16:26:53.606714 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 16:26:53.606714 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 16:26:53.606714 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 16:26:53.606714 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 16:26:53.606714 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 16:26:53.606714 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 16:26:53.606714 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application with Lock( Jul 01 16:26:53.606714 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 16:26:53.606714 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._enter() Jul 01 16:26:53.606714 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 16:26:53.606714 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 16:26:53.607679 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 16:26:53.607679 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.creator() Jul 01 16:26:53.607679 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 16:26:53.607679 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application created_value = creator( Jul 01 16:26:53.607679 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 01 16:26:53.607679 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 01 16:26:53.607679 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 01 16:26:53.607679 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 01 16:26:53.607679 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 01 16:26:53.607679 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 01 16:26:53.607679 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 01 16:26:53.607679 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Jul 01 16:26:53.609634 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 111/227] 10.222.0.57 () {56 vars in 1096 bytes} [Fri Jul 1 16:26:53 2022] GET /identity/v3/roles/admin => generated 83 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 16:26:53.610658 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2c293f-38fc-4d43-bd73-e7b26aa816f5 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:26:53.641024 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f59751c-bba4-41a4-b756-13bdadec4fc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:26:53.641856 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:26:53.642905 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f59751c-bba4-41a4-b756-13bdadec4fc4 None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:26:53.643367 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f59751c-bba4-41a4-b756-13bdadec4fc4 None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:26:53.643910 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f59751c-bba4-41a4-b756-13bdadec4fc4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:26:53.645943 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f59751c-bba4-41a4-b756-13bdadec4fc4 None admin] RBAC: Adding query filter params (role.id=983be40bdab04c478eaa90c6b6a46b14, scope.project.id=63f84f2aa0bf4707bd9551422f0cae3e, user.id=04bead9a70014c4e8222b6d571279eb3) {{(pid=80312) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:26:53.646459 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f59751c-bba4-41a4-b756-13bdadec4fc4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:26:53.648035 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f59751c-bba4-41a4-b756-13bdadec4fc4 None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:26:53.654549 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 117/228] 10.222.0.57 () {56 vars in 1374 bytes} [Fri Jul 1 16:26:53 2022] GET /identity/v3/role_assignments?user.id=04bead9a70014c4e8222b6d571279eb3&scope.project.id=63f84f2aa0bf4707bd9551422f0cae3e&role.id=983be40bdab04c478eaa90c6b6a46b14 => generated 259 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:26:53.662381 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f59751c-bba4-41a4-b756-13bdadec4fc4 None admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:26:53.720645 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2c293f-38fc-4d43-bd73-e7b26aa816f5 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:26:53.756588 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c170cfd7-a7b0-457b-b5f2-4522a4d892d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:26:53.757378 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c170cfd7-a7b0-457b-b5f2-4522a4d892d7 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:26:53.757667 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c170cfd7-a7b0-457b-b5f2-4522a4d892d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:26:53.757950 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c170cfd7-a7b0-457b-b5f2-4522a4d892d7 None admin] PATH_INFO: `/v3/projects/63f84f2aa0bf4707bd9551422f0cae3e` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:26:53.759188 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c170cfd7-a7b0-457b-b5f2-4522a4d892d7 None admin] RBAC: Authorizing `identity:get_project(project_id=63f84f2aa0bf4707bd9551422f0cae3e)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:26:53.761359 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c170cfd7-a7b0-457b-b5f2-4522a4d892d7 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:26:53.764013 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 112/229] 10.222.0.57 () {56 vars in 1186 bytes} [Fri Jul 1 16:26:53 2022] GET /identity/v3/projects/63f84f2aa0bf4707bd9551422f0cae3e => generated 302 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:26:53.771506 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c170cfd7-a7b0-457b-b5f2-4522a4d892d7 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:26:53.799397 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f59751c-bba4-41a4-b756-13bdadec4fc4 None admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:26:53.801509 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=100 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:26:53.851028 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04ac93b-2454-4e52-8a12-f550c4f24b6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:26:53.852082 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04ac93b-2454-4e52-8a12-f550c4f24b6c None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:26:53.852575 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04ac93b-2454-4e52-8a12-f550c4f24b6c None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:26:53.853057 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04ac93b-2454-4e52-8a12-f550c4f24b6c None admin] PATH_INFO: `/v3/roles` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:26:53.854742 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b04ac93b-2454-4e52-8a12-f550c4f24b6c None admin] RBAC: Adding query filter params (name=admin) {{(pid=80312) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:26:53.855394 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b04ac93b-2454-4e52-8a12-f550c4f24b6c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:26:53.857076 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b04ac93b-2454-4e52-8a12-f550c4f24b6c None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:26:53.867722 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 118/230] 10.222.0.57 () {56 vars in 1099 bytes} [Fri Jul 1 16:26:53 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:26:53.874592 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04ac93b-2454-4e52-8a12-f550c4f24b6c None admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:26:53.950716 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:26:53.957939 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c170cfd7-a7b0-457b-b5f2-4522a4d892d7 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:26:53.996893 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2324c36f-4136-44cf-9428-43f7b6e6caad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:26:53.997808 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2324c36f-4136-44cf-9428-43f7b6e6caad None admin] REQUEST_METHOD: `PUT` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:26:53.998092 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2324c36f-4136-44cf-9428-43f7b6e6caad None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:26:53.998381 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2324c36f-4136-44cf-9428-43f7b6e6caad None admin] PATH_INFO: `/v3/projects/63f84f2aa0bf4707bd9551422f0cae3e/users/6b9b08aa7935439db0311fdcd6ba14f8/roles/a8e5c73da81d4f0eb7b92c7058e500fe` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:26:54.001091 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2324c36f-4136-44cf-9428-43f7b6e6caad None admin] RBAC: Authorizing `identity:create_grant(project_id=63f84f2aa0bf4707bd9551422f0cae3e, user_id=6b9b08aa7935439db0311fdcd6ba14f8, role_id=a8e5c73da81d4f0eb7b92c7058e500fe)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:26:54.002871 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2324c36f-4136-44cf-9428-43f7b6e6caad None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:26:54.018176 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 113/231] 10.222.0.57 () {58 vars in 1439 bytes} [Fri Jul 1 16:26:53 2022] PUT /identity/v3/projects/63f84f2aa0bf4707bd9551422f0cae3e/users/6b9b08aa7935439db0311fdcd6ba14f8/roles/a8e5c73da81d4f0eb7b92c7058e500fe => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 16:26:54.050729 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04ac93b-2454-4e52-8a12-f550c4f24b6c None admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:26:54.090144 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48364fff-1244-40f7-9d4a-ca766f9ce40e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:26:54.091464 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48364fff-1244-40f7-9d4a-ca766f9ce40e None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:26:54.091946 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48364fff-1244-40f7-9d4a-ca766f9ce40e None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:26:54.092463 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48364fff-1244-40f7-9d4a-ca766f9ce40e None admin] PATH_INFO: `/v3/users/6b9b08aa7935439db0311fdcd6ba14f8` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:26:54.094450 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48364fff-1244-40f7-9d4a-ca766f9ce40e None admin] RBAC: Authorizing `identity:get_user(user_id=6b9b08aa7935439db0311fdcd6ba14f8)` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:26:54.096149 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48364fff-1244-40f7-9d4a-ca766f9ce40e None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:26:54.098063 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 119/232] 10.222.0.57 () {56 vars in 1177 bytes} [Fri Jul 1 16:26:53 2022] GET /identity/v3/users/6b9b08aa7935439db0311fdcd6ba14f8 => generated 283 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 01 16:26:54.103868 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2324c36f-4136-44cf-9428-43f7b6e6caad None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:26:54.253881 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2324c36f-4136-44cf-9428-43f7b6e6caad None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:26:54.289591 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5e3fd1-af04-45e8-91ce-2a14b65a58be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:26:54.290528 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5e3fd1-af04-45e8-91ce-2a14b65a58be None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:26:54.290817 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5e3fd1-af04-45e8-91ce-2a14b65a58be None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:26:54.291101 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5e3fd1-af04-45e8-91ce-2a14b65a58be None admin] PATH_INFO: `/v3/projects/63f84f2aa0bf4707bd9551422f0cae3e` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:26:54.292996 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae5e3fd1-af04-45e8-91ce-2a14b65a58be None admin] RBAC: Authorizing `identity:get_project(project_id=63f84f2aa0bf4707bd9551422f0cae3e)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:26:54.294575 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae5e3fd1-af04-45e8-91ce-2a14b65a58be None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:26:54.296697 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 114/233] 10.222.0.57 () {56 vars in 1186 bytes} [Fri Jul 1 16:26:54 2022] GET /identity/v3/projects/63f84f2aa0bf4707bd9551422f0cae3e => generated 302 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:26:54.302621 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48364fff-1244-40f7-9d4a-ca766f9ce40e None admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:26:54.339675 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=100 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:26:54.416023 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48364fff-1244-40f7-9d4a-ca766f9ce40e None admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:26:54.451113 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1c45b8-001d-4afc-b175-3c731b652a69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:26:54.452362 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1c45b8-001d-4afc-b175-3c731b652a69 None admin] REQUEST_METHOD: `PUT` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:26:54.452842 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1c45b8-001d-4afc-b175-3c731b652a69 None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:26:54.453317 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1c45b8-001d-4afc-b175-3c731b652a69 None admin] PATH_INFO: `/v3/projects/63f84f2aa0bf4707bd9551422f0cae3e/users/6b9b08aa7935439db0311fdcd6ba14f8/roles/84ee739146934c1ab32c501ff8093644` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:26:54.455858 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a1c45b8-001d-4afc-b175-3c731b652a69 None admin] RBAC: Authorizing `identity:create_grant(project_id=63f84f2aa0bf4707bd9551422f0cae3e, user_id=6b9b08aa7935439db0311fdcd6ba14f8, role_id=84ee739146934c1ab32c501ff8093644)` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:26:54.457523 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a1c45b8-001d-4afc-b175-3c731b652a69 None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:26:54.473388 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 120/234] 10.222.0.57 () {58 vars in 1439 bytes} [Fri Jul 1 16:26:54 2022] PUT /identity/v3/projects/63f84f2aa0bf4707bd9551422f0cae3e/users/6b9b08aa7935439db0311fdcd6ba14f8/roles/84ee739146934c1ab32c501ff8093644 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 16:26:54.983658 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9a8483-4c92-44cf-9cdc-b67df2674c2d None None] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:26:54.983977 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9a8483-4c92-44cf-9cdc-b67df2674c2d None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:26:54.984326 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9a8483-4c92-44cf-9cdc-b67df2674c2d None None] PATH_INFO: `/` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:26:54.985847 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 115/235] 10.222.0.57 () {54 vars in 926 bytes} [Fri Jul 1 16:26:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 16:26:54.991080 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b711b1-15d5-4403-b64e-7ad068f3c541 None None] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:26:54.991372 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b711b1-15d5-4403-b64e-7ad068f3c541 None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:26:54.991652 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b711b1-15d5-4403-b64e-7ad068f3c541 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:26:55.014250 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.auth.core [None req-08b711b1-15d5-4403-b64e-7ad068f3c541 None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80312) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:26:55.015470 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-08b711b1-15d5-4403-b64e-7ad068f3c541 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:26:55.115133 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 121/236] 10.222.0.57 () {58 vars in 1023 bytes} [Fri Jul 1 16:26:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 01 16:26:55.150627 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9a8483-4c92-44cf-9cdc-b67df2674c2d None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:26:55.175051 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdffcf32-1fbe-4143-9439-9d6d88d72e9c None None] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:26:55.175540 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdffcf32-1fbe-4143-9439-9d6d88d72e9c None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:26:55.175918 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdffcf32-1fbe-4143-9439-9d6d88d72e9c None None] PATH_INFO: `/` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:26:55.177305 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 122/237] 10.222.0.57 () {54 vars in 926 bytes} [Fri Jul 1 16:26:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 16:26:55.183002 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ce05e2-6331-4626-90e1-eccc54501217 None None] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:26:55.183376 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ce05e2-6331-4626-90e1-eccc54501217 None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:26:55.183663 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ce05e2-6331-4626-90e1-eccc54501217 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:26:55.190379 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:26:55.203193 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.auth.core [None req-e3ce05e2-6331-4626-90e1-eccc54501217 None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80312) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:26:55.204367 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-e3ce05e2-6331-4626-90e1-eccc54501217 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:26:55.246787 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:26:55.311718 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 123/238] 10.222.0.57 () {58 vars in 1023 bytes} [Fri Jul 1 16:26:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 01 16:26:55.317229 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9a8483-4c92-44cf-9cdc-b67df2674c2d None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:26:55.347801 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ce05e2-6331-4626-90e1-eccc54501217 None None] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:26:55.353197 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a866e9c4-d36a-4010-9db5-ebcba744a58b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:26:55.354602 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a866e9c4-d36a-4010-9db5-ebcba744a58b None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:26:55.354888 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a866e9c4-d36a-4010-9db5-ebcba744a58b None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:26:55.362473 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a866e9c4-d36a-4010-9db5-ebcba744a58b None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:26:55.366211 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a866e9c4-d36a-4010-9db5-ebcba744a58b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 01 16:26:55.374675 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application [None req-a866e9c4-d36a-4010-9db5-ebcba744a58b None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 01 16:26:55.374675 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 16:26:55.374675 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 16:26:55.374675 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 16:26:55.374675 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 16:26:55.374675 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 16:26:55.374675 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 16:26:55.374675 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 16:26:55.374675 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 16:26:55.374675 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 16:26:55.374675 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 16:26:55.374675 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 16:26:55.374675 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 01 16:26:55.374675 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 01 16:26:55.374675 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 01 16:26:55.374675 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 01 16:26:55.375614 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 01 16:26:55.375614 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 01 16:26:55.375614 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 01 16:26:55.375614 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 01 16:26:55.375614 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 16:26:55.375614 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 16:26:55.375614 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 16:26:55.375614 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 16:26:55.375614 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 16:26:55.375614 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 16:26:55.375614 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 16:26:55.375614 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application with Lock( Jul 01 16:26:55.375614 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 16:26:55.375614 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._enter() Jul 01 16:26:55.375614 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 16:26:55.375614 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 16:26:55.376413 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 16:26:55.376413 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.creator() Jul 01 16:26:55.376413 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 16:26:55.376413 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application created_value = creator( Jul 01 16:26:55.376413 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 01 16:26:55.376413 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 01 16:26:55.376413 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 01 16:26:55.376413 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 01 16:26:55.376413 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 01 16:26:55.376413 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 01 16:26:55.376413 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 01 16:26:55.376413 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Jul 01 16:26:55.376413 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 116/239] 10.222.0.57 () {56 vars in 1099 bytes} [Fri Jul 1 16:26:55 2022] GET /identity/v3/roles/reader => generated 84 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 16:26:55.396959 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a866e9c4-d36a-4010-9db5-ebcba744a58b None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:26:55.492494 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:26:55.521588 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a866e9c4-d36a-4010-9db5-ebcba744a58b None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:26:55.574207 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ce05e2-6331-4626-90e1-eccc54501217 None None] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:26:55.607428 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e294cff-781b-4d2d-bcc3-3dc3ac82c74b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:26:55.608485 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e294cff-781b-4d2d-bcc3-3dc3ac82c74b None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:26:55.609894 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e294cff-781b-4d2d-bcc3-3dc3ac82c74b None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:26:55.610472 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e294cff-781b-4d2d-bcc3-3dc3ac82c74b None admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:26:55.612136 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e294cff-781b-4d2d-bcc3-3dc3ac82c74b None admin] RBAC: Adding query filter params (name=reader) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:26:55.613974 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e294cff-781b-4d2d-bcc3-3dc3ac82c74b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:26:55.615600 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e294cff-781b-4d2d-bcc3-3dc3ac82c74b None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:26:55.629665 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef7e2b8-dbbd-414d-8cb9-a7bcd0d2c381 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:26:55.630820 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef7e2b8-dbbd-414d-8cb9-a7bcd0d2c381 None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:26:55.631320 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef7e2b8-dbbd-414d-8cb9-a7bcd0d2c381 None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:26:55.631804 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef7e2b8-dbbd-414d-8cb9-a7bcd0d2c381 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:26:55.633465 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef7e2b8-dbbd-414d-8cb9-a7bcd0d2c381 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:26:55.635711 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef7e2b8-dbbd-414d-8cb9-a7bcd0d2c381 None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:26:55.635944 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 117/240] 10.222.0.57 () {56 vars in 1101 bytes} [Fri Jul 1 16:26:55 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 01 16:26:55.636992 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e294cff-781b-4d2d-bcc3-3dc3ac82c74b None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:26:55.639485 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 124/241] 10.222.0.57 () {56 vars in 1108 bytes} [Fri Jul 1 16:26:55 2022] GET /identity/v3/domains/default => generated 199 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:26:55.644877 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef7e2b8-dbbd-414d-8cb9-a7bcd0d2c381 None admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:26:55.668369 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=100 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:26:55.804344 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef7e2b8-dbbd-414d-8cb9-a7bcd0d2c381 None admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:26:55.861537 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e294cff-781b-4d2d-bcc3-3dc3ac82c74b None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:26:55.889720 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5831da37-22a0-4841-8017-65aa27f7053c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:26:55.890338 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5831da37-22a0-4841-8017-65aa27f7053c None admin] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:26:55.890650 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5831da37-22a0-4841-8017-65aa27f7053c None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:26:55.890930 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5831da37-22a0-4841-8017-65aa27f7053c None admin] PATH_INFO: `/v3/users` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:26:55.891668 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5831da37-22a0-4841-8017-65aa27f7053c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:26:55.893524 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5831da37-22a0-4841-8017-65aa27f7053c None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:26:55.920782 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97118197-384c-45cc-b43c-115ad76e6ddf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:26:55.924119 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97118197-384c-45cc-b43c-115ad76e6ddf None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:26:55.927455 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97118197-384c-45cc-b43c-115ad76e6ddf None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:26:55.927983 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97118197-384c-45cc-b43c-115ad76e6ddf None admin] PATH_INFO: `/v3/users/504342654c3b4de58081127b593f0228` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:26:55.938730 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 125/242] 10.222.0.57 () {60 vars in 1132 bytes} [Fri Jul 1 16:26:55 2022] POST /identity/v3/users => generated 293 bytes in 294 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 16:26:55.961924 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97118197-384c-45cc-b43c-115ad76e6ddf None admin] RBAC: Authorizing `identity:get_user(user_id=504342654c3b4de58081127b593f0228)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:26:55.963974 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97118197-384c-45cc-b43c-115ad76e6ddf None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:26:55.966132 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 118/243] 10.222.0.57 () {56 vars in 1177 bytes} [Fri Jul 1 16:26:55 2022] GET /identity/v3/users/504342654c3b4de58081127b593f0228 => generated 297 bytes in 329 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:26:55.971812 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5831da37-22a0-4841-8017-65aa27f7053c None admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:26:56.065917 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:26:56.092647 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5831da37-22a0-4841-8017-65aa27f7053c None admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:26:56.128974 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255919a6-97a8-462b-9912-c4729a7eb6fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:26:56.129634 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255919a6-97a8-462b-9912-c4729a7eb6fb None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:26:56.129910 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255919a6-97a8-462b-9912-c4729a7eb6fb None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:26:56.130189 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255919a6-97a8-462b-9912-c4729a7eb6fb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:26:56.149694 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4481e607-989f-46ed-ae44-b79b09a390cd None None] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:26:56.149977 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4481e607-989f-46ed-ae44-b79b09a390cd None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:26:56.150156 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4481e607-989f-46ed-ae44-b79b09a390cd None None] PATH_INFO: `/` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:26:56.152369 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 119/244] 10.222.0.57 () {54 vars in 926 bytes} [Fri Jul 1 16:26:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 01 16:26:56.158379 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6427a467-e299-4958-9230-2fb24cb89f73 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:26:56.158665 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6427a467-e299-4958-9230-2fb24cb89f73 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:26:56.158951 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6427a467-e299-4958-9230-2fb24cb89f73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:26:56.165880 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:26:56.179792 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-6427a467-e299-4958-9230-2fb24cb89f73 None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:26:56.181607 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-6427a467-e299-4958-9230-2fb24cb89f73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:26:56.239343 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-255919a6-97a8-462b-9912-c4729a7eb6fb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:26:56.240840 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-255919a6-97a8-462b-9912-c4729a7eb6fb None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:26:56.267887 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 120/245] 10.222.0.57 () {58 vars in 1023 bytes} [Fri Jul 1 16:26:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 01 16:26:56.298049 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6427a467-e299-4958-9230-2fb24cb89f73 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:26:56.334708 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=100 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:26:56.359828 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 126/246] 10.222.0.57 () {58 vars in 1282 bytes} [Fri Jul 1 16:26:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 388 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 01 16:26:56.365453 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255919a6-97a8-462b-9912-c4729a7eb6fb None admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:26:56.426011 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6427a467-e299-4958-9230-2fb24cb89f73 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:26:56.466410 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b0281b-103f-47ac-bcdd-0d53d9568a1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:26:56.467174 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b0281b-103f-47ac-bcdd-0d53d9568a1d None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:26:56.467455 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b0281b-103f-47ac-bcdd-0d53d9568a1d None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:26:56.467735 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b0281b-103f-47ac-bcdd-0d53d9568a1d None admin] PATH_INFO: `/v3/roles/member` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:26:56.470480 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:26:56.482793 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b0281b-103f-47ac-bcdd-0d53d9568a1d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 01 16:26:56.489578 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255919a6-97a8-462b-9912-c4729a7eb6fb None admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:26:56.500350 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application [None req-a6b0281b-103f-47ac-bcdd-0d53d9568a1d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 01 16:26:56.500350 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 16:26:56.500350 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 16:26:56.500350 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 16:26:56.500350 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 16:26:56.500350 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 16:26:56.500350 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 16:26:56.500350 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 16:26:56.500350 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 16:26:56.500350 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 16:26:56.500350 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 16:26:56.500350 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 16:26:56.500350 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 01 16:26:56.500350 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 01 16:26:56.500350 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 01 16:26:56.500350 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 01 16:26:56.501350 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 01 16:26:56.501350 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 01 16:26:56.501350 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 01 16:26:56.501350 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 01 16:26:56.501350 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 16:26:56.501350 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 16:26:56.501350 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 16:26:56.501350 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 16:26:56.501350 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 16:26:56.501350 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 16:26:56.501350 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 16:26:56.501350 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application with Lock( Jul 01 16:26:56.501350 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 16:26:56.501350 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._enter() Jul 01 16:26:56.501350 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 16:26:56.501350 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 16:26:56.502142 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 16:26:56.502142 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.creator() Jul 01 16:26:56.502142 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 16:26:56.502142 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application created_value = creator( Jul 01 16:26:56.502142 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 01 16:26:56.502142 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 01 16:26:56.502142 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 01 16:26:56.502142 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 01 16:26:56.502142 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 01 16:26:56.502142 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 01 16:26:56.502142 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 01 16:26:56.502142 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Jul 01 16:26:56.502142 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 121/247] 10.222.0.57 () {56 vars in 1099 bytes} [Fri Jul 1 16:26:56 2022] GET /identity/v3/roles/member => generated 84 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 16:26:56.508592 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b0281b-103f-47ac-bcdd-0d53d9568a1d None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:26:56.529407 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95842a2-15f6-4ecd-90fb-f15fc7c63a18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:26:56.530527 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95842a2-15f6-4ecd-90fb-f15fc7c63a18 None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:26:56.531200 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95842a2-15f6-4ecd-90fb-f15fc7c63a18 None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:26:56.531494 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95842a2-15f6-4ecd-90fb-f15fc7c63a18 None admin] PATH_INFO: `/v3/projects/63f84f2aa0bf4707bd9551422f0cae3e` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:26:56.533009 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b95842a2-15f6-4ecd-90fb-f15fc7c63a18 None admin] RBAC: Authorizing `identity:get_project(project_id=63f84f2aa0bf4707bd9551422f0cae3e)` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:26:56.535108 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b95842a2-15f6-4ecd-90fb-f15fc7c63a18 None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:26:56.536918 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 127/248] 10.222.0.57 () {56 vars in 1186 bytes} [Fri Jul 1 16:26:56 2022] GET /identity/v3/projects/63f84f2aa0bf4707bd9551422f0cae3e => generated 302 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 01 16:26:56.543505 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95842a2-15f6-4ecd-90fb-f15fc7c63a18 None admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:26:56.594730 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=100 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:26:56.629037 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b0281b-103f-47ac-bcdd-0d53d9568a1d None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:26:56.665175 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95842a2-15f6-4ecd-90fb-f15fc7c63a18 None admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:26:56.677156 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99a5247-916c-41ee-99bc-1528c289bc81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:26:56.677806 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99a5247-916c-41ee-99bc-1528c289bc81 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:26:56.678101 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99a5247-916c-41ee-99bc-1528c289bc81 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:26:56.678414 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99a5247-916c-41ee-99bc-1528c289bc81 None admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:26:56.679286 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d99a5247-916c-41ee-99bc-1528c289bc81 None admin] RBAC: Adding query filter params (name=member) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:26:56.679605 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d99a5247-916c-41ee-99bc-1528c289bc81 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:26:56.681113 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d99a5247-916c-41ee-99bc-1528c289bc81 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:26:56.691900 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 122/249] 10.222.0.57 () {56 vars in 1101 bytes} [Fri Jul 1 16:26:56 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:26:56.698082 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99a5247-916c-41ee-99bc-1528c289bc81 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:26:56.704379 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18df4a6-110d-4cc6-a69e-d5eb462ec8e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:26:56.705053 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18df4a6-110d-4cc6-a69e-d5eb462ec8e9 None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:26:56.705588 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18df4a6-110d-4cc6-a69e-d5eb462ec8e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:26:56.705879 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18df4a6-110d-4cc6-a69e-d5eb462ec8e9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:26:56.706862 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c18df4a6-110d-4cc6-a69e-d5eb462ec8e9 None admin] RBAC: Adding query filter params (role.id=bd7cefce73d14bd7827b58b6bfbe5e7a, scope.project.id=63f84f2aa0bf4707bd9551422f0cae3e, user.id=504342654c3b4de58081127b593f0228) {{(pid=80312) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:26:56.707183 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c18df4a6-110d-4cc6-a69e-d5eb462ec8e9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:26:56.708658 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c18df4a6-110d-4cc6-a69e-d5eb462ec8e9 None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:26:56.715613 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 128/250] 10.222.0.57 () {56 vars in 1374 bytes} [Fri Jul 1 16:26:56 2022] GET /identity/v3/role_assignments?user.id=504342654c3b4de58081127b593f0228&scope.project.id=63f84f2aa0bf4707bd9551422f0cae3e&role.id=bd7cefce73d14bd7827b58b6bfbe5e7a => generated 259 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 01 16:26:56.781033 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:26:56.800007 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8996d002-9093-409b-896e-428ee8dcf9e1 None None] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:26:56.800310 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8996d002-9093-409b-896e-428ee8dcf9e1 None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:26:56.800569 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8996d002-9093-409b-896e-428ee8dcf9e1 None None] PATH_INFO: `/` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:26:56.801966 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 129/251] 10.222.0.57 () {54 vars in 926 bytes} [Fri Jul 1 16:26:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 16:26:56.807790 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7173e359-49d1-4a62-bf0b-96659ab0c112 None None] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:26:56.808085 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7173e359-49d1-4a62-bf0b-96659ab0c112 None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:26:56.808352 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7173e359-49d1-4a62-bf0b-96659ab0c112 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:26:56.824095 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99a5247-916c-41ee-99bc-1528c289bc81 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:26:56.847889 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.auth.core [None req-7173e359-49d1-4a62-bf0b-96659ab0c112 None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80312) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:26:56.848641 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-7173e359-49d1-4a62-bf0b-96659ab0c112 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:26:56.879318 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c538e86-1942-4a73-a4f2-23135cdb769f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:26:56.880280 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c538e86-1942-4a73-a4f2-23135cdb769f None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:26:56.880569 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c538e86-1942-4a73-a4f2-23135cdb769f None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:26:56.880846 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c538e86-1942-4a73-a4f2-23135cdb769f None admin] PATH_INFO: `/v3/users/04bead9a70014c4e8222b6d571279eb3` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:26:56.883112 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c538e86-1942-4a73-a4f2-23135cdb769f None admin] RBAC: Authorizing `identity:get_user(user_id=04bead9a70014c4e8222b6d571279eb3)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:26:56.884696 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c538e86-1942-4a73-a4f2-23135cdb769f None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:26:56.887059 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 123/252] 10.222.0.57 () {56 vars in 1177 bytes} [Fri Jul 1 16:26:56 2022] GET /identity/v3/users/04bead9a70014c4e8222b6d571279eb3 => generated 297 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 01 16:26:56.893196 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c538e86-1942-4a73-a4f2-23135cdb769f None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:26:56.953549 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 130/253] 10.222.0.57 () {58 vars in 1023 bytes} [Fri Jul 1 16:26:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 01 16:26:56.986532 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7173e359-49d1-4a62-bf0b-96659ab0c112 None None] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:26:57.024452 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c538e86-1942-4a73-a4f2-23135cdb769f None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:26:57.033495 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=100 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:26:57.082747 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:26:57.103467 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7173e359-49d1-4a62-bf0b-96659ab0c112 None None] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:26:57.106275 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28eb8b6f-1e31-468b-bd64-c8311b1a477a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:26:57.107295 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28eb8b6f-1e31-468b-bd64-c8311b1a477a None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:26:57.107592 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28eb8b6f-1e31-468b-bd64-c8311b1a477a None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:26:57.107877 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28eb8b6f-1e31-468b-bd64-c8311b1a477a None admin] PATH_INFO: `/v3/projects/63f84f2aa0bf4707bd9551422f0cae3e` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:26:57.110899 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28eb8b6f-1e31-468b-bd64-c8311b1a477a None admin] RBAC: Authorizing `identity:get_project(project_id=63f84f2aa0bf4707bd9551422f0cae3e)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:26:57.112427 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28eb8b6f-1e31-468b-bd64-c8311b1a477a None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:26:57.114724 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 124/254] 10.222.0.57 () {56 vars in 1186 bytes} [Fri Jul 1 16:26:56 2022] GET /identity/v3/projects/63f84f2aa0bf4707bd9551422f0cae3e => generated 302 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 01 16:26:57.123606 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28eb8b6f-1e31-468b-bd64-c8311b1a477a None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:26:57.142715 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7fc0e62-5ca1-444b-9f91-2f1f1d244ae6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:26:57.143612 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7fc0e62-5ca1-444b-9f91-2f1f1d244ae6 None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:26:57.143960 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7fc0e62-5ca1-444b-9f91-2f1f1d244ae6 None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:26:57.144243 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7fc0e62-5ca1-444b-9f91-2f1f1d244ae6 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:26:57.157499 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7fc0e62-5ca1-444b-9f91-2f1f1d244ae6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 01 16:26:57.167013 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application [None req-c7fc0e62-5ca1-444b-9f91-2f1f1d244ae6 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 01 16:26:57.167013 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 16:26:57.167013 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 16:26:57.167013 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 16:26:57.167013 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 16:26:57.167013 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 16:26:57.167013 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 16:26:57.167013 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 16:26:57.167013 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 16:26:57.167013 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 16:26:57.167013 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 16:26:57.167013 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 16:26:57.167013 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 01 16:26:57.167013 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 01 16:26:57.167013 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 01 16:26:57.167013 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 01 16:26:57.169525 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 01 16:26:57.169525 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 01 16:26:57.169525 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 01 16:26:57.169525 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 01 16:26:57.169525 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 16:26:57.169525 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 16:26:57.169525 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 16:26:57.169525 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 16:26:57.169525 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 16:26:57.169525 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 16:26:57.169525 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 16:26:57.169525 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application with Lock( Jul 01 16:26:57.169525 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 16:26:57.169525 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self._enter() Jul 01 16:26:57.169525 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 16:26:57.169525 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 16:26:57.171466 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 16:26:57.171466 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.creator() Jul 01 16:26:57.171466 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 16:26:57.171466 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application created_value = creator( Jul 01 16:26:57.171466 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 01 16:26:57.171466 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 01 16:26:57.171466 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 01 16:26:57.171466 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 01 16:26:57.171466 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 01 16:26:57.171466 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 01 16:26:57.171466 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 01 16:26:57.171466 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application Jul 01 16:26:57.171466 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 131/255] 10.222.0.57 () {56 vars in 1114 bytes} [Fri Jul 1 16:26:56 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 01 16:26:57.175041 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7fc0e62-5ca1-444b-9f91-2f1f1d244ae6 None admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:26:57.294511 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=100 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:26:57.305895 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7fc0e62-5ca1-444b-9f91-2f1f1d244ae6 None admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:26:57.312296 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28eb8b6f-1e31-468b-bd64-c8311b1a477a None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:26:57.342481 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fed66af-f6ed-4964-b666-6aa3913629aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:26:57.343283 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fed66af-f6ed-4964-b666-6aa3913629aa None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:26:57.343574 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fed66af-f6ed-4964-b666-6aa3913629aa None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:26:57.343851 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fed66af-f6ed-4964-b666-6aa3913629aa None admin] PATH_INFO: `/v3/roles` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:26:57.344606 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fed66af-f6ed-4964-b666-6aa3913629aa None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80312) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:26:57.344917 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fed66af-f6ed-4964-b666-6aa3913629aa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:26:57.346313 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fed66af-f6ed-4964-b666-6aa3913629aa None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:26:57.355595 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 132/256] 10.222.0.57 () {56 vars in 1111 bytes} [Fri Jul 1 16:26:57 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:26:57.356683 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee608dd-a882-40a0-b1c3-e18367e3441f None None] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:26:57.356972 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee608dd-a882-40a0-b1c3-e18367e3441f None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:26:57.357246 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee608dd-a882-40a0-b1c3-e18367e3441f None None] PATH_INFO: `/` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:26:57.358588 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 133/257] 10.222.0.57 () {54 vars in 926 bytes} [Fri Jul 1 16:26:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 16:26:57.364055 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee608dd-a882-40a0-b1c3-e18367e3441f None None] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:26:57.386332 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8fd1de-c621-4960-a00c-7abb4c633a81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:26:57.387251 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8fd1de-c621-4960-a00c-7abb4c633a81 None admin] REQUEST_METHOD: `PUT` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:26:57.387533 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8fd1de-c621-4960-a00c-7abb4c633a81 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:26:57.387809 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8fd1de-c621-4960-a00c-7abb4c633a81 None admin] PATH_INFO: `/v3/projects/63f84f2aa0bf4707bd9551422f0cae3e/users/04bead9a70014c4e8222b6d571279eb3/roles/983be40bdab04c478eaa90c6b6a46b14` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:26:57.394252 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b8fd1de-c621-4960-a00c-7abb4c633a81 None admin] RBAC: Authorizing `identity:create_grant(project_id=63f84f2aa0bf4707bd9551422f0cae3e, user_id=04bead9a70014c4e8222b6d571279eb3, role_id=983be40bdab04c478eaa90c6b6a46b14)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:26:57.395732 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b8fd1de-c621-4960-a00c-7abb4c633a81 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:26:57.423431 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 125/258] 10.222.0.57 () {58 vars in 1439 bytes} [Fri Jul 1 16:26:57 2022] PUT /identity/v3/projects/63f84f2aa0bf4707bd9551422f0cae3e/users/04bead9a70014c4e8222b6d571279eb3/roles/983be40bdab04c478eaa90c6b6a46b14 => generated 0 bytes in 303 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 01 16:26:57.424889 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894fecdf-f475-4ee8-8eff-17156cfdb8de None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:26:57.425185 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894fecdf-f475-4ee8-8eff-17156cfdb8de None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:26:57.425468 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894fecdf-f475-4ee8-8eff-17156cfdb8de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:26:57.456394 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-894fecdf-f475-4ee8-8eff-17156cfdb8de None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:26:57.460768 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-894fecdf-f475-4ee8-8eff-17156cfdb8de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:26:57.573885 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:26:57.583585 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee608dd-a882-40a0-b1c3-e18367e3441f None None] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:26:57.618667 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 126/259] 10.222.0.57 () {58 vars in 1023 bytes} [Fri Jul 1 16:26:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 01 16:26:57.644663 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de87550c-f5c2-42d4-bbe1-942edc8a2ed5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:26:57.645973 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de87550c-f5c2-42d4-bbe1-942edc8a2ed5 None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:26:57.646523 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de87550c-f5c2-42d4-bbe1-942edc8a2ed5 None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:26:57.647006 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de87550c-f5c2-42d4-bbe1-942edc8a2ed5 None admin] PATH_INFO: `/v3/users/6b9b08aa7935439db0311fdcd6ba14f8` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:26:57.649691 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de87550c-f5c2-42d4-bbe1-942edc8a2ed5 None admin] RBAC: Authorizing `identity:get_user(user_id=6b9b08aa7935439db0311fdcd6ba14f8)` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:26:57.651900 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de87550c-f5c2-42d4-bbe1-942edc8a2ed5 None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:26:57.653983 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 134/260] 10.222.0.57 () {56 vars in 1177 bytes} [Fri Jul 1 16:26:57 2022] GET /identity/v3/users/6b9b08aa7935439db0311fdcd6ba14f8 => generated 283 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:26:57.660744 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894fecdf-f475-4ee8-8eff-17156cfdb8de None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:26:57.664331 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de87550c-f5c2-42d4-bbe1-942edc8a2ed5 None admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:26:57.719522 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=100 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:26:57.850828 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de87550c-f5c2-42d4-bbe1-942edc8a2ed5 None admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:26:57.908148 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcbd6ee4-76de-45d3-92cf-aa2aac9a915e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:26:57.908948 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbd6ee4-76de-45d3-92cf-aa2aac9a915e None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:26:57.909230 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbd6ee4-76de-45d3-92cf-aa2aac9a915e None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:26:57.909502 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbd6ee4-76de-45d3-92cf-aa2aac9a915e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:26:57.927476 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcbd6ee4-76de-45d3-92cf-aa2aac9a915e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 01 16:26:57.936811 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application [None req-dcbd6ee4-76de-45d3-92cf-aa2aac9a915e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 01 16:26:57.936811 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 16:26:57.936811 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 16:26:57.936811 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 16:26:57.936811 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 16:26:57.936811 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 16:26:57.936811 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 16:26:57.936811 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 16:26:57.936811 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 16:26:57.936811 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 16:26:57.936811 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 16:26:57.936811 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 16:26:57.936811 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 01 16:26:57.936811 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 01 16:26:57.936811 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 01 16:26:57.936811 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 01 16:26:57.937767 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 01 16:26:57.937767 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 01 16:26:57.937767 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 01 16:26:57.937767 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 01 16:26:57.937767 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 16:26:57.937767 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 16:26:57.937767 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 16:26:57.937767 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 16:26:57.937767 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 16:26:57.937767 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 16:26:57.937767 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 16:26:57.937767 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application with Lock( Jul 01 16:26:57.937767 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 16:26:57.937767 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self._enter() Jul 01 16:26:57.937767 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 16:26:57.937767 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 16:26:57.938585 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 16:26:57.938585 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.creator() Jul 01 16:26:57.938585 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 16:26:57.938585 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application created_value = creator( Jul 01 16:26:57.938585 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 01 16:26:57.938585 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 01 16:26:57.938585 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 01 16:26:57.938585 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 01 16:26:57.938585 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 01 16:26:57.938585 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 01 16:26:57.938585 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 01 16:26:57.938585 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application Jul 01 16:26:57.938585 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 135/261] 10.222.0.57 () {56 vars in 1096 bytes} [Fri Jul 1 16:26:57 2022] GET /identity/v3/roles/admin => generated 83 bytes in 274 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 16:26:57.944004 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcbd6ee4-76de-45d3-92cf-aa2aac9a915e None admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:26:57.949028 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894fecdf-f475-4ee8-8eff-17156cfdb8de None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:26:58.078658 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcbd6ee4-76de-45d3-92cf-aa2aac9a915e None admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:26:58.090704 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=100 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:26:58.119206 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518b3f7b-a95c-494f-93dd-9e252d525c13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:26:58.119826 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518b3f7b-a95c-494f-93dd-9e252d525c13 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:26:58.120112 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518b3f7b-a95c-494f-93dd-9e252d525c13 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:26:58.120392 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518b3f7b-a95c-494f-93dd-9e252d525c13 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:26:58.131902 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0968a6b9-fed4-48ec-b12e-691503b87fe6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:26:58.132909 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0968a6b9-fed4-48ec-b12e-691503b87fe6 None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:26:58.133244 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0968a6b9-fed4-48ec-b12e-691503b87fe6 None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:26:58.133524 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0968a6b9-fed4-48ec-b12e-691503b87fe6 None admin] PATH_INFO: `/v3/roles` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:26:58.134336 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0968a6b9-fed4-48ec-b12e-691503b87fe6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80312) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:26:58.134665 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0968a6b9-fed4-48ec-b12e-691503b87fe6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:26:58.136709 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0968a6b9-fed4-48ec-b12e-691503b87fe6 None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:26:58.152389 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 136/262] 10.222.0.57 () {56 vars in 1099 bytes} [Fri Jul 1 16:26:57 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 01 16:26:58.158060 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0968a6b9-fed4-48ec-b12e-691503b87fe6 None admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:26:58.317518 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0968a6b9-fed4-48ec-b12e-691503b87fe6 None admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:26:58.331964 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:26:58.366330 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9721e986-abd2-4f09-9762-a68281e92f31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:26:58.367392 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9721e986-abd2-4f09-9762-a68281e92f31 None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:26:58.367683 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9721e986-abd2-4f09-9762-a68281e92f31 None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:26:58.367958 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9721e986-abd2-4f09-9762-a68281e92f31 None admin] PATH_INFO: `/v3/users/6b9b08aa7935439db0311fdcd6ba14f8` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:26:58.371375 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9721e986-abd2-4f09-9762-a68281e92f31 None admin] RBAC: Authorizing `identity:get_user(user_id=6b9b08aa7935439db0311fdcd6ba14f8)` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:26:58.372782 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9721e986-abd2-4f09-9762-a68281e92f31 None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:26:58.374638 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 137/263] 10.222.0.57 () {56 vars in 1177 bytes} [Fri Jul 1 16:26:58 2022] GET /identity/v3/users/6b9b08aa7935439db0311fdcd6ba14f8 => generated 283 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:26:58.380490 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9721e986-abd2-4f09-9762-a68281e92f31 None admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:26:58.414746 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=100 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:26:58.431200 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-518b3f7b-a95c-494f-93dd-9e252d525c13 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:26:58.433257 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-518b3f7b-a95c-494f-93dd-9e252d525c13 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:26:58.515713 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9721e986-abd2-4f09-9762-a68281e92f31 None admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:26:58.572139 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55025cb-bb49-483f-a5fa-1d84fac11b70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:26:58.572878 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55025cb-bb49-483f-a5fa-1d84fac11b70 None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:26:58.573161 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55025cb-bb49-483f-a5fa-1d84fac11b70 None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:26:58.573435 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55025cb-bb49-483f-a5fa-1d84fac11b70 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:26:58.649379 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a55025cb-bb49-483f-a5fa-1d84fac11b70 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:26:58.651537 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a55025cb-bb49-483f-a5fa-1d84fac11b70 None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:26:58.714844 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=100 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:26:58.760288 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 127/264] 10.222.0.57 () {58 vars in 1282 bytes} [Fri Jul 1 16:26:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 1100 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 01 16:26:58.769846 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518b3f7b-a95c-494f-93dd-9e252d525c13 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:26:58.791905 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 138/265] 10.222.0.57 () {58 vars in 1282 bytes} [Fri Jul 1 16:26:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 408 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 01 16:26:58.794030 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55025cb-bb49-483f-a5fa-1d84fac11b70 None admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:26:58.906017 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:26:58.938082 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518b3f7b-a95c-494f-93dd-9e252d525c13 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:26:59.032348 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55025cb-bb49-483f-a5fa-1d84fac11b70 None admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:26:59.048836 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50b3662-6901-478e-8a21-4fa744271ed6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:26:59.049719 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50b3662-6901-478e-8a21-4fa744271ed6 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:26:59.050006 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50b3662-6901-478e-8a21-4fa744271ed6 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:26:59.050284 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50b3662-6901-478e-8a21-4fa744271ed6 None admin] PATH_INFO: `/v3/projects/63f84f2aa0bf4707bd9551422f0cae3e` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:26:59.051655 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a50b3662-6901-478e-8a21-4fa744271ed6 None admin] RBAC: Authorizing `identity:get_project(project_id=63f84f2aa0bf4707bd9551422f0cae3e)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:26:59.054877 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a50b3662-6901-478e-8a21-4fa744271ed6 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:26:59.054996 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 128/266] 10.222.0.57 () {56 vars in 1186 bytes} [Fri Jul 1 16:26:58 2022] GET /identity/v3/projects/63f84f2aa0bf4707bd9551422f0cae3e => generated 302 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 01 16:26:59.061167 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50b3662-6901-478e-8a21-4fa744271ed6 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:26:59.079586 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=100 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:26:59.110445 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c796c72f-03bd-4b19-ba53-aff93b15897f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:26:59.111328 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c796c72f-03bd-4b19-ba53-aff93b15897f None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:26:59.111615 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c796c72f-03bd-4b19-ba53-aff93b15897f None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:26:59.111894 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c796c72f-03bd-4b19-ba53-aff93b15897f None admin] PATH_INFO: `/v3/projects/63f84f2aa0bf4707bd9551422f0cae3e` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:26:59.115290 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c796c72f-03bd-4b19-ba53-aff93b15897f None admin] RBAC: Authorizing `identity:get_project(project_id=63f84f2aa0bf4707bd9551422f0cae3e)` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:26:59.116768 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c796c72f-03bd-4b19-ba53-aff93b15897f None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:26:59.118716 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 139/267] 10.222.0.57 () {56 vars in 1186 bytes} [Fri Jul 1 16:26:58 2022] GET /identity/v3/projects/63f84f2aa0bf4707bd9551422f0cae3e => generated 302 bytes in 325 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 01 16:26:59.127758 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c796c72f-03bd-4b19-ba53-aff93b15897f None admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:26:59.316459 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c796c72f-03bd-4b19-ba53-aff93b15897f None admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:26:59.341479 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50b3662-6901-478e-8a21-4fa744271ed6 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:26:59.381182 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff8c9b5-3b85-4e53-b36d-45c5f4359f0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:26:59.381959 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff8c9b5-3b85-4e53-b36d-45c5f4359f0d None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:26:59.382249 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff8c9b5-3b85-4e53-b36d-45c5f4359f0d None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:26:59.382566 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff8c9b5-3b85-4e53-b36d-45c5f4359f0d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:26:59.383405 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff8c9b5-3b85-4e53-b36d-45c5f4359f0d None admin] RBAC: Adding query filter params (role.id=84ee739146934c1ab32c501ff8093644, scope.project.id=63f84f2aa0bf4707bd9551422f0cae3e, user.id=6b9b08aa7935439db0311fdcd6ba14f8) {{(pid=80312) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:26:59.383815 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff8c9b5-3b85-4e53-b36d-45c5f4359f0d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:26:59.385413 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff8c9b5-3b85-4e53-b36d-45c5f4359f0d None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:26:59.398161 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 140/268] 10.222.0.57 () {56 vars in 1374 bytes} [Fri Jul 1 16:26:59 2022] GET /identity/v3/role_assignments?user.id=6b9b08aa7935439db0311fdcd6ba14f8&scope.project.id=63f84f2aa0bf4707bd9551422f0cae3e&role.id=84ee739146934c1ab32c501ff8093644 => generated 608 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 01 16:26:59.403133 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bebe998a-c1c3-487f-8cfc-869c6d6d83b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:26:59.403813 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebe998a-c1c3-487f-8cfc-869c6d6d83b6 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:26:59.404094 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebe998a-c1c3-487f-8cfc-869c6d6d83b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:26:59.404370 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebe998a-c1c3-487f-8cfc-869c6d6d83b6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:26:59.405218 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bebe998a-c1c3-487f-8cfc-869c6d6d83b6 None admin] RBAC: Adding query filter params (role.id=a8e5c73da81d4f0eb7b92c7058e500fe, scope.project.id=63f84f2aa0bf4707bd9551422f0cae3e, user.id=6b9b08aa7935439db0311fdcd6ba14f8) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:26:59.405531 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bebe998a-c1c3-487f-8cfc-869c6d6d83b6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:26:59.407009 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bebe998a-c1c3-487f-8cfc-869c6d6d83b6 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:26:59.414007 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 129/269] 10.222.0.57 () {56 vars in 1374 bytes} [Fri Jul 1 16:26:59 2022] GET /identity/v3/role_assignments?user.id=6b9b08aa7935439db0311fdcd6ba14f8&scope.project.id=63f84f2aa0bf4707bd9551422f0cae3e&role.id=a8e5c73da81d4f0eb7b92c7058e500fe => generated 608 bytes in 353 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:26:59.487389 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad14008e-7225-4e9a-ad5e-1c0b594dcfa2 None None] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:26:59.488075 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad14008e-7225-4e9a-ad5e-1c0b594dcfa2 None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:26:59.488557 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad14008e-7225-4e9a-ad5e-1c0b594dcfa2 None None] PATH_INFO: `/` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:26:59.490596 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 141/270] 10.222.0.57 () {54 vars in 926 bytes} [Fri Jul 1 16:26:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 16:26:59.496858 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa48af0-633b-4b7c-99db-771eb33623a4 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:26:59.497149 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa48af0-633b-4b7c-99db-771eb33623a4 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:26:59.497419 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa48af0-633b-4b7c-99db-771eb33623a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:26:59.523551 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-0fa48af0-633b-4b7c-99db-771eb33623a4 None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:26:59.524806 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-0fa48af0-633b-4b7c-99db-771eb33623a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:26:59.537289 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:26:59.628028 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 130/271] 10.222.0.57 () {58 vars in 1023 bytes} [Fri Jul 1 16:26:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 01 16:26:59.628447 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97f0fb0-1d77-4348-b92a-595b148b2bda None None] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:26:59.628737 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97f0fb0-1d77-4348-b92a-595b148b2bda None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:26:59.629016 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97f0fb0-1d77-4348-b92a-595b148b2bda None None] PATH_INFO: `/` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:26:59.630769 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 142/272] 10.222.0.57 () {54 vars in 926 bytes} [Fri Jul 1 16:26:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 16:26:59.645988 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910c0719-c033-4ec0-be87-775ff1403b9e None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:26:59.646646 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910c0719-c033-4ec0-be87-775ff1403b9e None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:26:59.647134 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910c0719-c033-4ec0-be87-775ff1403b9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:26:59.671701 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-910c0719-c033-4ec0-be87-775ff1403b9e None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:26:59.673239 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-910c0719-c033-4ec0-be87-775ff1403b9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:26:59.694995 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97f0fb0-1d77-4348-b92a-595b148b2bda None None] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:26:59.759097 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 131/273] 10.222.0.57 () {58 vars in 1023 bytes} [Fri Jul 1 16:26:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 01 16:26:59.766860 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=100 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:26:59.794978 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a36a92-da91-48f0-990f-6b7d2511c935 None None] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:26:59.795596 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a36a92-da91-48f0-990f-6b7d2511c935 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:26:59.796097 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a36a92-da91-48f0-990f-6b7d2511c935 None None] PATH_INFO: `/` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:26:59.798017 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 132/274] 10.222.0.57 () {54 vars in 926 bytes} [Fri Jul 1 16:26:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 01 16:26:59.799654 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a36a92-da91-48f0-990f-6b7d2511c935 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:26:59.839454 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97f0fb0-1d77-4348-b92a-595b148b2bda None None] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:26:59.892133 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e371d6c-01ad-411d-aefc-866017232bac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:26:59.892868 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e371d6c-01ad-411d-aefc-866017232bac None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:26:59.893151 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e371d6c-01ad-411d-aefc-866017232bac None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:26:59.893427 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e371d6c-01ad-411d-aefc-866017232bac None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:26:59.896896 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:26:59.912367 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e371d6c-01ad-411d-aefc-866017232bac None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 01 16:26:59.921564 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application [None req-7e371d6c-01ad-411d-aefc-866017232bac None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 01 16:26:59.921564 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 16:26:59.921564 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 16:26:59.921564 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 16:26:59.921564 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 16:26:59.921564 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 16:26:59.921564 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 16:26:59.921564 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 16:26:59.921564 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 16:26:59.921564 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 16:26:59.921564 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 16:26:59.921564 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 16:26:59.921564 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 01 16:26:59.921564 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 01 16:26:59.921564 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 01 16:26:59.921564 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 01 16:26:59.922535 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 01 16:26:59.922535 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 01 16:26:59.922535 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 01 16:26:59.922535 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 01 16:26:59.922535 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 16:26:59.922535 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 16:26:59.922535 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 16:26:59.922535 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 16:26:59.922535 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 16:26:59.922535 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 16:26:59.922535 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 16:26:59.922535 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application with Lock( Jul 01 16:26:59.922535 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 16:26:59.922535 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self._enter() Jul 01 16:26:59.922535 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 16:26:59.922535 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 16:26:59.923329 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 16:26:59.923329 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.creator() Jul 01 16:26:59.923329 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 16:26:59.923329 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application created_value = creator( Jul 01 16:26:59.923329 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 01 16:26:59.923329 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 01 16:26:59.923329 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 01 16:26:59.923329 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 01 16:26:59.923329 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 01 16:26:59.923329 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 01 16:26:59.923329 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 01 16:26:59.923329 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application Jul 01 16:26:59.923329 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 143/275] 10.222.0.57 () {56 vars in 1099 bytes} [Fri Jul 1 16:26:59 2022] GET /identity/v3/roles/reader => generated 84 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 16:26:59.924550 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4881e72-c897-4f0e-96a4-8d09c9b140bb None None] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:26:59.924837 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4881e72-c897-4f0e-96a4-8d09c9b140bb None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:26:59.925119 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4881e72-c897-4f0e-96a4-8d09c9b140bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:26:59.935836 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a36a92-da91-48f0-990f-6b7d2511c935 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:26:59.944250 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.auth.core [None req-f4881e72-c897-4f0e-96a4-8d09c9b140bb None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80312) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:26:59.945437 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-f4881e72-c897-4f0e-96a4-8d09c9b140bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:26:59.973181 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1937617-08ab-4cac-9810-c08eb149a19a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:26:59.973974 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1937617-08ab-4cac-9810-c08eb149a19a None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:26:59.974258 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1937617-08ab-4cac-9810-c08eb149a19a None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:26:59.974721 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1937617-08ab-4cac-9810-c08eb149a19a None admin] PATH_INFO: `/v3/domains/default` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:26:59.975744 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1937617-08ab-4cac-9810-c08eb149a19a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:26:59.977197 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1937617-08ab-4cac-9810-c08eb149a19a None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:26:59.984434 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 133/276] 10.222.0.57 () {56 vars in 1108 bytes} [Fri Jul 1 16:26:59 2022] GET /identity/v3/domains/default => generated 199 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:26:59.984434 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1937617-08ab-4cac-9810-c08eb149a19a None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:27:00.021685 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 144/277] 10.222.0.57 () {58 vars in 1023 bytes} [Fri Jul 1 16:26:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 01 16:27:00.022679 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4881e72-c897-4f0e-96a4-8d09c9b140bb None None] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:27:00.067695 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=100 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:27:00.106375 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1937617-08ab-4cac-9810-c08eb149a19a None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:27:00.133273 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4881e72-c897-4f0e-96a4-8d09c9b140bb None None] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:27:00.142285 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821bbc5e-59e4-4b43-9ccb-4f62a543050f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:27:00.142916 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821bbc5e-59e4-4b43-9ccb-4f62a543050f None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:00.143202 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821bbc5e-59e4-4b43-9ccb-4f62a543050f None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:00.143491 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821bbc5e-59e4-4b43-9ccb-4f62a543050f None admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:00.144243 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-821bbc5e-59e4-4b43-9ccb-4f62a543050f None admin] RBAC: Adding query filter params (name=reader) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:27:00.144556 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-821bbc5e-59e4-4b43-9ccb-4f62a543050f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:27:00.145945 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-821bbc5e-59e4-4b43-9ccb-4f62a543050f None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:27:00.159745 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 134/278] 10.222.0.57 () {56 vars in 1101 bytes} [Fri Jul 1 16:26:59 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:27:00.159745 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821bbc5e-59e4-4b43-9ccb-4f62a543050f None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:27:00.162468 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:27:00.176195 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc260fc9-5187-4713-9003-a064a5fea303 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:27:00.176875 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc260fc9-5187-4713-9003-a064a5fea303 None admin] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:00.177160 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc260fc9-5187-4713-9003-a064a5fea303 None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:00.177437 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc260fc9-5187-4713-9003-a064a5fea303 None admin] PATH_INFO: `/v3/users` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:00.178144 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc260fc9-5187-4713-9003-a064a5fea303 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:27:00.179630 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc260fc9-5187-4713-9003-a064a5fea303 None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:27:00.210050 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 145/279] 10.222.0.57 () {60 vars in 1132 bytes} [Fri Jul 1 16:27:00 2022] POST /identity/v3/users => generated 293 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 16:27:00.210050 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc260fc9-5187-4713-9003-a064a5fea303 None admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:27:00.282875 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821bbc5e-59e4-4b43-9ccb-4f62a543050f None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:27:00.336671 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfefe827-3121-4312-9d19-1bbd44fbfe67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:27:00.338084 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfefe827-3121-4312-9d19-1bbd44fbfe67 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:00.338494 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfefe827-3121-4312-9d19-1bbd44fbfe67 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:00.338883 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfefe827-3121-4312-9d19-1bbd44fbfe67 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:00.357559 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfefe827-3121-4312-9d19-1bbd44fbfe67 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 01 16:27:00.360212 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:27:00.365943 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application [None req-bfefe827-3121-4312-9d19-1bbd44fbfe67 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 01 16:27:00.365943 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 16:27:00.365943 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 16:27:00.365943 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 16:27:00.365943 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 16:27:00.365943 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 16:27:00.365943 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 16:27:00.365943 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 16:27:00.365943 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 16:27:00.365943 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 16:27:00.365943 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 16:27:00.365943 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 16:27:00.365943 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 01 16:27:00.365943 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 01 16:27:00.365943 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 01 16:27:00.365943 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 01 16:27:00.367546 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 01 16:27:00.367546 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 01 16:27:00.367546 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 01 16:27:00.367546 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 01 16:27:00.367546 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 16:27:00.367546 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 16:27:00.367546 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 16:27:00.367546 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 16:27:00.367546 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 16:27:00.367546 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 16:27:00.367546 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 16:27:00.367546 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application with Lock( Jul 01 16:27:00.367546 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 16:27:00.367546 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._enter() Jul 01 16:27:00.367546 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 16:27:00.367546 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 16:27:00.368676 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 16:27:00.368676 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.creator() Jul 01 16:27:00.368676 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 16:27:00.368676 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application created_value = creator( Jul 01 16:27:00.368676 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 01 16:27:00.368676 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 01 16:27:00.368676 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 01 16:27:00.368676 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 01 16:27:00.368676 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 01 16:27:00.368676 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 01 16:27:00.368676 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 01 16:27:00.368676 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Jul 01 16:27:00.374340 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 135/280] 10.222.0.57 () {56 vars in 1099 bytes} [Fri Jul 1 16:27:00 2022] GET /identity/v3/roles/member => generated 84 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 16:27:00.376598 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfefe827-3121-4312-9d19-1bbd44fbfe67 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:27:00.385459 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc260fc9-5187-4713-9003-a064a5fea303 None admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:27:00.441418 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692aa68c-c1d8-40c4-b524-0036f61a9365 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:27:00.442299 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692aa68c-c1d8-40c4-b524-0036f61a9365 None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:00.442768 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692aa68c-c1d8-40c4-b524-0036f61a9365 None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:00.443045 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692aa68c-c1d8-40c4-b524-0036f61a9365 None admin] PATH_INFO: `/v3/users/504342654c3b4de58081127b593f0228` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:00.445338 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-692aa68c-c1d8-40c4-b524-0036f61a9365 None admin] RBAC: Authorizing `identity:get_user(user_id=504342654c3b4de58081127b593f0228)` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:27:00.446171 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-692aa68c-c1d8-40c4-b524-0036f61a9365 None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:27:00.448101 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 146/281] 10.222.0.57 () {56 vars in 1177 bytes} [Fri Jul 1 16:27:00 2022] GET /identity/v3/users/504342654c3b4de58081127b593f0228 => generated 297 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:27:00.449562 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4c7082-7391-4b0d-99df-1dcfd9626e14 None None] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:00.449857 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4c7082-7391-4b0d-99df-1dcfd9626e14 None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:00.450136 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4c7082-7391-4b0d-99df-1dcfd9626e14 None None] PATH_INFO: `/` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:00.451612 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 147/282] 10.222.0.57 () {54 vars in 926 bytes} [Fri Jul 1 16:27:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 16:27:00.454393 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4c7082-7391-4b0d-99df-1dcfd9626e14 None None] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:27:00.470236 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:27:00.521390 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfefe827-3121-4312-9d19-1bbd44fbfe67 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:27:00.566055 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db1efda-0ebb-41db-8f97-0515127dce80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:27:00.566856 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db1efda-0ebb-41db-8f97-0515127dce80 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:00.567145 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db1efda-0ebb-41db-8f97-0515127dce80 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:00.567425 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db1efda-0ebb-41db-8f97-0515127dce80 None admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:00.568190 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db1efda-0ebb-41db-8f97-0515127dce80 None admin] RBAC: Adding query filter params (name=member) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:27:00.568502 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db1efda-0ebb-41db-8f97-0515127dce80 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:27:00.569890 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db1efda-0ebb-41db-8f97-0515127dce80 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:27:00.581722 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 136/283] 10.222.0.57 () {56 vars in 1101 bytes} [Fri Jul 1 16:27:00 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 01 16:27:00.583368 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d824312-7869-4933-b08c-212b5441bf0e None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:00.583660 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d824312-7869-4933-b08c-212b5441bf0e None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:00.583934 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d824312-7869-4933-b08c-212b5441bf0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:00.591123 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4c7082-7391-4b0d-99df-1dcfd9626e14 None None] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:27:00.602502 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-8d824312-7869-4933-b08c-212b5441bf0e None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:27:00.604457 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-8d824312-7869-4933-b08c-212b5441bf0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:27:00.631723 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca89d87-a03e-4dcb-b0ea-323da14840f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:27:00.632695 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca89d87-a03e-4dcb-b0ea-323da14840f9 None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:00.633047 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca89d87-a03e-4dcb-b0ea-323da14840f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:00.633412 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca89d87-a03e-4dcb-b0ea-323da14840f9 None admin] PATH_INFO: `/v3/projects/63f84f2aa0bf4707bd9551422f0cae3e` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:00.635305 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca89d87-a03e-4dcb-b0ea-323da14840f9 None admin] RBAC: Authorizing `identity:get_project(project_id=63f84f2aa0bf4707bd9551422f0cae3e)` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:27:00.636849 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca89d87-a03e-4dcb-b0ea-323da14840f9 None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:27:00.638653 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 148/284] 10.222.0.57 () {56 vars in 1186 bytes} [Fri Jul 1 16:27:00 2022] GET /identity/v3/projects/63f84f2aa0bf4707bd9551422f0cae3e => generated 302 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 01 16:27:00.639436 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca89d87-a03e-4dcb-b0ea-323da14840f9 None admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:27:00.681188 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=100 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:27:00.697954 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 137/285] 10.222.0.57 () {58 vars in 1023 bytes} [Fri Jul 1 16:27:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 01 16:27:00.699205 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d824312-7869-4933-b08c-212b5441bf0e None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:27:00.761607 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca89d87-a03e-4dcb-b0ea-323da14840f9 None admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:27:00.778949 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:27:00.798726 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2181f4e-2bdd-47cf-a71f-875cb0403162 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:27:00.799613 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2181f4e-2bdd-47cf-a71f-875cb0403162 None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:00.799910 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2181f4e-2bdd-47cf-a71f-875cb0403162 None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:00.800159 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2181f4e-2bdd-47cf-a71f-875cb0403162 None admin] PATH_INFO: `/v3/users/6cd5ade2cd254dda8ebb77652bcba049` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:00.814972 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2181f4e-2bdd-47cf-a71f-875cb0403162 None admin] RBAC: Authorizing `identity:get_user(user_id=6cd5ade2cd254dda8ebb77652bcba049)` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:27:00.816751 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2181f4e-2bdd-47cf-a71f-875cb0403162 None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:27:00.818197 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 149/286] 10.222.0.57 () {56 vars in 1177 bytes} [Fri Jul 1 16:27:00 2022] GET /identity/v3/users/6cd5ade2cd254dda8ebb77652bcba049 => generated 293 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:27:00.819007 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2181f4e-2bdd-47cf-a71f-875cb0403162 None admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:27:00.859099 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d824312-7869-4933-b08c-212b5441bf0e None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:27:00.895760 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afdad0f9-5465-4257-8882-a8326b81b66c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:27:00.896923 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdad0f9-5465-4257-8882-a8326b81b66c None admin] REQUEST_METHOD: `PUT` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:00.897369 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdad0f9-5465-4257-8882-a8326b81b66c None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:00.897782 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdad0f9-5465-4257-8882-a8326b81b66c None admin] PATH_INFO: `/v3/projects/63f84f2aa0bf4707bd9551422f0cae3e/users/504342654c3b4de58081127b593f0228/roles/bd7cefce73d14bd7827b58b6bfbe5e7a` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:00.899937 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afdad0f9-5465-4257-8882-a8326b81b66c None admin] RBAC: Authorizing `identity:create_grant(project_id=63f84f2aa0bf4707bd9551422f0cae3e, user_id=504342654c3b4de58081127b593f0228, role_id=bd7cefce73d14bd7827b58b6bfbe5e7a)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:27:00.902334 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afdad0f9-5465-4257-8882-a8326b81b66c None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:27:00.926704 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 138/287] 10.222.0.57 () {58 vars in 1439 bytes} [Fri Jul 1 16:27:00 2022] PUT /identity/v3/projects/63f84f2aa0bf4707bd9551422f0cae3e/users/504342654c3b4de58081127b593f0228/roles/bd7cefce73d14bd7827b58b6bfbe5e7a => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 16:27:00.927137 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=100 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:27:00.927934 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afdad0f9-5465-4257-8882-a8326b81b66c None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:27:00.950082 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2181f4e-2bdd-47cf-a71f-875cb0403162 None admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:27:01.020975 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3413e60-6523-4c5f-b6a2-b3d7f3ef29aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:27:01.023315 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3413e60-6523-4c5f-b6a2-b3d7f3ef29aa None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:01.023669 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3413e60-6523-4c5f-b6a2-b3d7f3ef29aa None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:01.023878 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3413e60-6523-4c5f-b6a2-b3d7f3ef29aa None admin] PATH_INFO: `/v3/roles/member` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:01.048771 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3413e60-6523-4c5f-b6a2-b3d7f3ef29aa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 01 16:27:01.063772 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application [None req-f3413e60-6523-4c5f-b6a2-b3d7f3ef29aa None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 01 16:27:01.063772 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 16:27:01.063772 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 16:27:01.063772 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 16:27:01.063772 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 16:27:01.063772 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 16:27:01.063772 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 16:27:01.063772 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 16:27:01.063772 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 16:27:01.063772 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 16:27:01.063772 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 16:27:01.063772 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 16:27:01.063772 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 01 16:27:01.063772 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 01 16:27:01.063772 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 01 16:27:01.063772 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 01 16:27:01.064743 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 01 16:27:01.064743 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 01 16:27:01.064743 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 01 16:27:01.064743 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 01 16:27:01.064743 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 16:27:01.064743 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 16:27:01.064743 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 16:27:01.064743 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 16:27:01.064743 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 16:27:01.064743 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 16:27:01.064743 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 16:27:01.064743 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application with Lock( Jul 01 16:27:01.064743 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 16:27:01.064743 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self._enter() Jul 01 16:27:01.064743 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 16:27:01.064743 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 16:27:01.065539 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 16:27:01.065539 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.creator() Jul 01 16:27:01.065539 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 16:27:01.065539 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application created_value = creator( Jul 01 16:27:01.065539 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 01 16:27:01.065539 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 01 16:27:01.065539 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 01 16:27:01.065539 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 01 16:27:01.065539 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 01 16:27:01.065539 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 01 16:27:01.065539 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 01 16:27:01.065539 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application Jul 01 16:27:01.067439 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 150/288] 10.222.0.57 () {56 vars in 1099 bytes} [Fri Jul 1 16:27:00 2022] GET /identity/v3/roles/member => generated 84 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 16:27:01.074325 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3413e60-6523-4c5f-b6a2-b3d7f3ef29aa None admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:27:01.135062 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afdad0f9-5465-4257-8882-a8326b81b66c None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:27:01.162735 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:27:01.193853 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf5487e-68c2-4a83-a60e-80919eec4f5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:27:01.194588 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf5487e-68c2-4a83-a60e-80919eec4f5c None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:01.194874 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf5487e-68c2-4a83-a60e-80919eec4f5c None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:01.195156 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf5487e-68c2-4a83-a60e-80919eec4f5c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:01.196888 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf5487e-68c2-4a83-a60e-80919eec4f5c None admin] RBAC: Adding query filter params (role.id=983be40bdab04c478eaa90c6b6a46b14, user.id=6cd5ade2cd254dda8ebb77652bcba049, scope.system=all) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:27:01.197282 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf5487e-68c2-4a83-a60e-80919eec4f5c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:27:01.199206 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf5487e-68c2-4a83-a60e-80919eec4f5c None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:27:01.207011 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 139/289] 10.222.0.57 () {56 vars in 1308 bytes} [Fri Jul 1 16:27:00 2022] GET /identity/v3/role_assignments?user.id=6cd5ade2cd254dda8ebb77652bcba049&scope.system=all&role.id=983be40bdab04c478eaa90c6b6a46b14 => generated 226 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:27:01.251943 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3413e60-6523-4c5f-b6a2-b3d7f3ef29aa None admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:27:01.305026 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bcebb2f-7a51-47a0-8a58-d03fc1d25a79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:27:01.305726 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcebb2f-7a51-47a0-8a58-d03fc1d25a79 None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:01.306009 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcebb2f-7a51-47a0-8a58-d03fc1d25a79 None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:01.306284 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcebb2f-7a51-47a0-8a58-d03fc1d25a79 None admin] PATH_INFO: `/v3/roles` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:01.308082 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bcebb2f-7a51-47a0-8a58-d03fc1d25a79 None admin] RBAC: Adding query filter params (name=member) {{(pid=80312) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:27:01.308446 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bcebb2f-7a51-47a0-8a58-d03fc1d25a79 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:27:01.309887 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bcebb2f-7a51-47a0-8a58-d03fc1d25a79 None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:27:01.323309 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 151/290] 10.222.0.57 () {56 vars in 1101 bytes} [Fri Jul 1 16:27:01 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:27:01.328740 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf5487e-68c2-4a83-a60e-80919eec4f5c None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:27:01.451246 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf5487e-68c2-4a83-a60e-80919eec4f5c None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:27:01.504141 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9da0673-0e34-4463-8675-f39c5b8fdc86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:27:01.505611 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9da0673-0e34-4463-8675-f39c5b8fdc86 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:01.506066 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9da0673-0e34-4463-8675-f39c5b8fdc86 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:01.506411 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9da0673-0e34-4463-8675-f39c5b8fdc86 None admin] PATH_INFO: `/v3/users/04bead9a70014c4e8222b6d571279eb3` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:01.508083 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9da0673-0e34-4463-8675-f39c5b8fdc86 None admin] RBAC: Authorizing `identity:get_user(user_id=04bead9a70014c4e8222b6d571279eb3)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:27:01.509563 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9da0673-0e34-4463-8675-f39c5b8fdc86 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:27:01.511741 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 140/291] 10.222.0.57 () {56 vars in 1177 bytes} [Fri Jul 1 16:27:01 2022] GET /identity/v3/users/04bead9a70014c4e8222b6d571279eb3 => generated 297 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:27:01.517855 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bcebb2f-7a51-47a0-8a58-d03fc1d25a79 None admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:27:01.536823 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=100 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:27:01.633631 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bcebb2f-7a51-47a0-8a58-d03fc1d25a79 None admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:27:01.670290 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd44ea0d-2d03-4eec-9115-9e9b4fa5270a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:27:01.670986 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd44ea0d-2d03-4eec-9115-9e9b4fa5270a None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:01.671338 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd44ea0d-2d03-4eec-9115-9e9b4fa5270a None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:01.671627 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd44ea0d-2d03-4eec-9115-9e9b4fa5270a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:01.747158 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd44ea0d-2d03-4eec-9115-9e9b4fa5270a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:27:01.748658 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd44ea0d-2d03-4eec-9115-9e9b4fa5270a None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:27:01.763599 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=100 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:27:01.860245 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 152/292] 10.222.0.57 () {58 vars in 1282 bytes} [Fri Jul 1 16:27:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 01 16:27:01.867437 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9da0673-0e34-4463-8675-f39c5b8fdc86 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:27:01.946688 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:27:02.005390 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9da0673-0e34-4463-8675-f39c5b8fdc86 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:27:02.051048 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4abd5132-e33a-4b2f-aef1-2a66e33d4977 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:27:02.051923 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abd5132-e33a-4b2f-aef1-2a66e33d4977 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:02.052208 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abd5132-e33a-4b2f-aef1-2a66e33d4977 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:02.052488 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abd5132-e33a-4b2f-aef1-2a66e33d4977 None admin] PATH_INFO: `/v3/projects/63f84f2aa0bf4707bd9551422f0cae3e` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:02.053699 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4abd5132-e33a-4b2f-aef1-2a66e33d4977 None admin] RBAC: Authorizing `identity:get_project(project_id=63f84f2aa0bf4707bd9551422f0cae3e)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:27:02.055170 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4abd5132-e33a-4b2f-aef1-2a66e33d4977 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:27:02.057799 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 141/293] 10.222.0.57 () {56 vars in 1186 bytes} [Fri Jul 1 16:27:01 2022] GET /identity/v3/projects/63f84f2aa0bf4707bd9551422f0cae3e => generated 302 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:27:02.063566 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd44ea0d-2d03-4eec-9115-9e9b4fa5270a None admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:27:02.211445 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd44ea0d-2d03-4eec-9115-9e9b4fa5270a None admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:27:02.233182 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=100 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:27:02.260992 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6361344f-0dc2-4243-8663-bfce7b552364 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:27:02.261670 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6361344f-0dc2-4243-8663-bfce7b552364 None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:02.261955 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6361344f-0dc2-4243-8663-bfce7b552364 None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:02.262238 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6361344f-0dc2-4243-8663-bfce7b552364 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:02.263120 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6361344f-0dc2-4243-8663-bfce7b552364 None admin] RBAC: Adding query filter params (role.id=983be40bdab04c478eaa90c6b6a46b14, scope.project.id=63f84f2aa0bf4707bd9551422f0cae3e, user.id=04bead9a70014c4e8222b6d571279eb3) {{(pid=80312) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:27:02.263443 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6361344f-0dc2-4243-8663-bfce7b552364 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:27:02.264916 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6361344f-0dc2-4243-8663-bfce7b552364 None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:27:02.274961 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 153/294] 10.222.0.57 () {56 vars in 1374 bytes} [Fri Jul 1 16:27:02 2022] GET /identity/v3/role_assignments?user.id=04bead9a70014c4e8222b6d571279eb3&scope.project.id=63f84f2aa0bf4707bd9551422f0cae3e&role.id=983be40bdab04c478eaa90c6b6a46b14 => generated 608 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:27:02.755228 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc58bc3-b63c-4f3b-b61c-22138a1f56f0 None None] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:02.755749 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc58bc3-b63c-4f3b-b61c-22138a1f56f0 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:02.755826 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc58bc3-b63c-4f3b-b61c-22138a1f56f0 None None] PATH_INFO: `/` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:02.757418 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 142/295] 10.222.0.57 () {54 vars in 926 bytes} [Fri Jul 1 16:27:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 16:27:02.762749 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a77148-e96b-4369-92f3-4e7bf10dc869 None None] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:02.763168 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a77148-e96b-4369-92f3-4e7bf10dc869 None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:02.763339 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a77148-e96b-4369-92f3-4e7bf10dc869 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:02.781074 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.auth.core [None req-16a77148-e96b-4369-92f3-4e7bf10dc869 None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80312) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:27:02.782277 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-16a77148-e96b-4369-92f3-4e7bf10dc869 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:27:02.864942 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 154/296] 10.222.0.57 () {58 vars in 1023 bytes} [Fri Jul 1 16:27:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 01 16:27:02.889211 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc58bc3-b63c-4f3b-b61c-22138a1f56f0 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:27:02.933030 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47fb45f-7c4f-4842-bdfc-620bc9e1c62c None None] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:02.933339 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47fb45f-7c4f-4842-bdfc-620bc9e1c62c None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:02.933608 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47fb45f-7c4f-4842-bdfc-620bc9e1c62c None None] PATH_INFO: `/` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:02.935314 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 155/297] 10.222.0.57 () {54 vars in 926 bytes} [Fri Jul 1 16:27:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 01 16:27:02.940734 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3b490c-2efb-484c-b00d-cc92c86343b2 None None] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:02.941033 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3b490c-2efb-484c-b00d-cc92c86343b2 None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:02.941310 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3b490c-2efb-484c-b00d-cc92c86343b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:02.962653 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.auth.core [None req-2a3b490c-2efb-484c-b00d-cc92c86343b2 None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80312) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:27:02.964226 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-2a3b490c-2efb-484c-b00d-cc92c86343b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:27:03.014876 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc58bc3-b63c-4f3b-b61c-22138a1f56f0 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:27:03.033157 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:27:03.052488 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 156/298] 10.222.0.57 () {58 vars in 1023 bytes} [Fri Jul 1 16:27:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 01 16:27:03.059080 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcee81e8-9cd4-4d57-afcf-4f448d8393a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:27:03.060071 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcee81e8-9cd4-4d57-afcf-4f448d8393a8 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:03.060545 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcee81e8-9cd4-4d57-afcf-4f448d8393a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:03.060962 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcee81e8-9cd4-4d57-afcf-4f448d8393a8 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:03.062329 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcee81e8-9cd4-4d57-afcf-4f448d8393a8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:27:03.063960 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcee81e8-9cd4-4d57-afcf-4f448d8393a8 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:27:03.066215 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 143/299] 10.222.0.57 () {56 vars in 1108 bytes} [Fri Jul 1 16:27:02 2022] GET /identity/v3/domains/default => generated 199 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:27:03.071449 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3b490c-2efb-484c-b00d-cc92c86343b2 None None] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:27:03.077403 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcee81e8-9cd4-4d57-afcf-4f448d8393a8 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:27:03.117060 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=100 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:27:03.194369 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3b490c-2efb-484c-b00d-cc92c86343b2 None None] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:27:03.212744 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcee81e8-9cd4-4d57-afcf-4f448d8393a8 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:27:03.234823 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb23b57-b0ff-4152-b38d-535dbd9fdfce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:27:03.235638 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb23b57-b0ff-4152-b38d-535dbd9fdfce None admin] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:03.236040 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb23b57-b0ff-4152-b38d-535dbd9fdfce None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:03.236659 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb23b57-b0ff-4152-b38d-535dbd9fdfce None admin] PATH_INFO: `/v3/groups` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:03.237835 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb23b57-b0ff-4152-b38d-535dbd9fdfce None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:27:03.239413 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb23b57-b0ff-4152-b38d-535dbd9fdfce None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:27:03.258872 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441573da-fd72-4983-be10-f14a2c67dfbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:27:03.259223 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 157/300] 10.222.0.57 () {60 vars in 1134 bytes} [Fri Jul 1 16:27:03 2022] POST /identity/v3/groups => generated 229 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 16:27:03.260777 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441573da-fd72-4983-be10-f14a2c67dfbe None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:03.261330 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441573da-fd72-4983-be10-f14a2c67dfbe None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:03.261882 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441573da-fd72-4983-be10-f14a2c67dfbe None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:03.280969 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-441573da-fd72-4983-be10-f14a2c67dfbe None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 01 16:27:03.290433 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application [None req-441573da-fd72-4983-be10-f14a2c67dfbe None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 01 16:27:03.290433 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 16:27:03.290433 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 16:27:03.290433 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 16:27:03.290433 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 16:27:03.290433 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 16:27:03.290433 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 16:27:03.290433 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 16:27:03.290433 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 16:27:03.290433 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 16:27:03.290433 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 16:27:03.290433 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 16:27:03.290433 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 01 16:27:03.290433 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 01 16:27:03.290433 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 01 16:27:03.290433 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 01 16:27:03.291626 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 01 16:27:03.291626 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 01 16:27:03.291626 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 01 16:27:03.291626 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 01 16:27:03.291626 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 16:27:03.291626 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 16:27:03.291626 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 16:27:03.291626 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 16:27:03.291626 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 16:27:03.291626 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 16:27:03.291626 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 16:27:03.291626 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application with Lock( Jul 01 16:27:03.291626 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 16:27:03.291626 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._enter() Jul 01 16:27:03.291626 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 16:27:03.291626 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 16:27:03.292450 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 16:27:03.292450 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.creator() Jul 01 16:27:03.292450 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 16:27:03.292450 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application created_value = creator( Jul 01 16:27:03.292450 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 01 16:27:03.292450 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 01 16:27:03.292450 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 01 16:27:03.292450 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 01 16:27:03.292450 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 01 16:27:03.292450 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 01 16:27:03.292450 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 01 16:27:03.292450 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Jul 01 16:27:03.295127 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 144/301] 10.222.0.57 () {56 vars in 1099 bytes} [Fri Jul 1 16:27:03 2022] GET /identity/v3/roles/reader => generated 84 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 01 16:27:03.301796 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb23b57-b0ff-4152-b38d-535dbd9fdfce None admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:27:03.420988 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:27:03.430299 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb23b57-b0ff-4152-b38d-535dbd9fdfce None admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:27:03.469359 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31fe09d-2c4e-4419-840c-54362577280e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:27:03.470035 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31fe09d-2c4e-4419-840c-54362577280e None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:03.470315 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31fe09d-2c4e-4419-840c-54362577280e None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:03.470613 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31fe09d-2c4e-4419-840c-54362577280e None admin] PATH_INFO: `/v3/roles` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:03.471366 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e31fe09d-2c4e-4419-840c-54362577280e None admin] RBAC: Adding query filter params (name=reader) {{(pid=80312) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:27:03.471667 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e31fe09d-2c4e-4419-840c-54362577280e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:27:03.473049 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e31fe09d-2c4e-4419-840c-54362577280e None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:27:03.482484 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 158/302] 10.222.0.57 () {56 vars in 1101 bytes} [Fri Jul 1 16:27:03 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:27:03.488110 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441573da-fd72-4983-be10-f14a2c67dfbe None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:27:03.543258 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:27:03.610551 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441573da-fd72-4983-be10-f14a2c67dfbe None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:27:03.648116 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4603888d-ef02-46b0-8312-bdfba841dfc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:27:03.649005 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4603888d-ef02-46b0-8312-bdfba841dfc2 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:03.649295 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4603888d-ef02-46b0-8312-bdfba841dfc2 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:03.649575 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4603888d-ef02-46b0-8312-bdfba841dfc2 None admin] PATH_INFO: `/v3/users/9f0dcfde5e924ebcb328cef131b4bb34` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:03.664975 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4603888d-ef02-46b0-8312-bdfba841dfc2 None admin] RBAC: Authorizing `identity:get_user(user_id=9f0dcfde5e924ebcb328cef131b4bb34)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:27:03.666710 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4603888d-ef02-46b0-8312-bdfba841dfc2 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:27:03.668763 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 145/303] 10.222.0.57 () {56 vars in 1177 bytes} [Fri Jul 1 16:27:03 2022] GET /identity/v3/users/9f0dcfde5e924ebcb328cef131b4bb34 => generated 293 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:27:03.674600 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31fe09d-2c4e-4419-840c-54362577280e None admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:27:03.686026 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf3fc09-cd88-4a67-8b7d-5532496f4b26 None None] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:03.686324 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf3fc09-cd88-4a67-8b7d-5532496f4b26 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:03.686627 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf3fc09-cd88-4a67-8b7d-5532496f4b26 None None] PATH_INFO: `/` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:03.688272 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 146/304] 10.222.0.57 () {54 vars in 926 bytes} [Fri Jul 1 16:27:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 16:27:03.693277 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad804a2-fa8c-4e9c-9622-09f8bd7f77aa None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:03.693574 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad804a2-fa8c-4e9c-9622-09f8bd7f77aa None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:03.693860 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad804a2-fa8c-4e9c-9622-09f8bd7f77aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:03.711816 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-3ad804a2-fa8c-4e9c-9622-09f8bd7f77aa None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:27:03.713549 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-3ad804a2-fa8c-4e9c-9622-09f8bd7f77aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:27:03.785160 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 147/305] 10.222.0.57 () {58 vars in 1023 bytes} [Fri Jul 1 16:27:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 01 16:27:03.786759 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230ca566-afdd-4ba5-9798-79b409d0bb12 None None] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:03.787115 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230ca566-afdd-4ba5-9798-79b409d0bb12 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:03.787460 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230ca566-afdd-4ba5-9798-79b409d0bb12 None None] PATH_INFO: `/` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:03.789041 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 148/306] 10.222.0.57 () {54 vars in 926 bytes} [Fri Jul 1 16:27:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 16:27:03.795320 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e67b03-d91a-4cc8-8468-1fbe8d86046a None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:03.795584 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e67b03-d91a-4cc8-8468-1fbe8d86046a None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:03.795845 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e67b03-d91a-4cc8-8468-1fbe8d86046a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:03.802407 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31fe09d-2c4e-4419-840c-54362577280e None admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:27:03.814963 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-95e67b03-d91a-4cc8-8468-1fbe8d86046a None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:27:03.816148 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-95e67b03-d91a-4cc8-8468-1fbe8d86046a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:27:03.848095 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:27:03.859327 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180ce274-8030-4719-b99f-0cc18ba63b78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:27:03.861774 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180ce274-8030-4719-b99f-0cc18ba63b78 None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:03.862327 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180ce274-8030-4719-b99f-0cc18ba63b78 None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:03.862805 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180ce274-8030-4719-b99f-0cc18ba63b78 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:03.865451 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-180ce274-8030-4719-b99f-0cc18ba63b78 None admin] RBAC: Adding query filter params (role.id=bd7cefce73d14bd7827b58b6bfbe5e7a, user.id=9f0dcfde5e924ebcb328cef131b4bb34, scope.system=all) {{(pid=80312) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:27:03.865876 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-180ce274-8030-4719-b99f-0cc18ba63b78 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:27:03.870535 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-180ce274-8030-4719-b99f-0cc18ba63b78 None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:27:03.878197 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 159/307] 10.222.0.57 () {56 vars in 1308 bytes} [Fri Jul 1 16:27:03 2022] GET /identity/v3/role_assignments?user.id=9f0dcfde5e924ebcb328cef131b4bb34&scope.system=all&role.id=bd7cefce73d14bd7827b58b6bfbe5e7a => generated 226 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:27:03.878885 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180ce274-8030-4719-b99f-0cc18ba63b78 None admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:27:03.913428 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=100 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:27:03.923305 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 149/308] 10.222.0.57 () {58 vars in 1023 bytes} [Fri Jul 1 16:27:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 01 16:27:03.950522 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e67b03-d91a-4cc8-8468-1fbe8d86046a None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:27:04.034094 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180ce274-8030-4719-b99f-0cc18ba63b78 None admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:27:04.074694 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e67b03-d91a-4cc8-8468-1fbe8d86046a None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:27:04.078534 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa134d97-634a-42e3-9113-0c34b881a1ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:27:04.079131 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa134d97-634a-42e3-9113-0c34b881a1ec None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:04.079419 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa134d97-634a-42e3-9113-0c34b881a1ec None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:04.079701 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa134d97-634a-42e3-9113-0c34b881a1ec None admin] PATH_INFO: `/v3/roles/member` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:04.092995 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa134d97-634a-42e3-9113-0c34b881a1ec None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 01 16:27:04.101376 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application [None req-fa134d97-634a-42e3-9113-0c34b881a1ec None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 01 16:27:04.101376 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 16:27:04.101376 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 16:27:04.101376 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 16:27:04.101376 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 16:27:04.101376 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 16:27:04.101376 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 16:27:04.101376 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 16:27:04.101376 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 16:27:04.101376 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 16:27:04.101376 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 16:27:04.101376 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 16:27:04.101376 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 01 16:27:04.101376 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 01 16:27:04.101376 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 01 16:27:04.101376 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 01 16:27:04.102308 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 01 16:27:04.102308 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 01 16:27:04.102308 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 01 16:27:04.102308 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 01 16:27:04.102308 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 16:27:04.102308 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 16:27:04.102308 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 16:27:04.102308 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 16:27:04.102308 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 16:27:04.102308 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 16:27:04.102308 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 16:27:04.102308 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application with Lock( Jul 01 16:27:04.102308 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 16:27:04.102308 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self._enter() Jul 01 16:27:04.102308 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 16:27:04.102308 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 16:27:04.103208 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 16:27:04.103208 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.creator() Jul 01 16:27:04.103208 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 16:27:04.103208 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application created_value = creator( Jul 01 16:27:04.103208 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 01 16:27:04.103208 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 01 16:27:04.103208 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 01 16:27:04.103208 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 01 16:27:04.103208 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 01 16:27:04.103208 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 01 16:27:04.103208 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 01 16:27:04.103208 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application Jul 01 16:27:04.103208 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 160/309] 10.222.0.57 () {56 vars in 1099 bytes} [Fri Jul 1 16:27:03 2022] GET /identity/v3/roles/member => generated 84 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 16:27:04.116149 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa134d97-634a-42e3-9113-0c34b881a1ec None admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:27:04.120877 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c70dfe5-66d9-45bf-a75f-92234bb10e48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:27:04.122501 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c70dfe5-66d9-45bf-a75f-92234bb10e48 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:04.122501 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c70dfe5-66d9-45bf-a75f-92234bb10e48 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:04.122679 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c70dfe5-66d9-45bf-a75f-92234bb10e48 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:04.137806 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c70dfe5-66d9-45bf-a75f-92234bb10e48 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 01 16:27:04.146870 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application [None req-5c70dfe5-66d9-45bf-a75f-92234bb10e48 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 01 16:27:04.146870 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 16:27:04.146870 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 16:27:04.146870 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 16:27:04.146870 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 16:27:04.146870 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 16:27:04.146870 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 16:27:04.146870 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 16:27:04.146870 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 16:27:04.146870 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 16:27:04.146870 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 16:27:04.146870 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 16:27:04.146870 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 01 16:27:04.146870 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 01 16:27:04.146870 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 01 16:27:04.146870 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 01 16:27:04.148457 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 01 16:27:04.148457 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 01 16:27:04.148457 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 01 16:27:04.148457 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 01 16:27:04.148457 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 16:27:04.148457 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 16:27:04.148457 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 16:27:04.148457 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 16:27:04.148457 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 16:27:04.148457 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 16:27:04.148457 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 16:27:04.148457 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application with Lock( Jul 01 16:27:04.148457 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 16:27:04.148457 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._enter() Jul 01 16:27:04.148457 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 16:27:04.148457 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 16:27:04.149933 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 16:27:04.149933 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.creator() Jul 01 16:27:04.149933 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 16:27:04.149933 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application created_value = creator( Jul 01 16:27:04.149933 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 01 16:27:04.149933 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 01 16:27:04.149933 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 01 16:27:04.149933 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 01 16:27:04.149933 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 01 16:27:04.149933 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 01 16:27:04.149933 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 01 16:27:04.149933 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Jul 01 16:27:04.154618 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 150/310] 10.222.0.57 () {56 vars in 1099 bytes} [Fri Jul 1 16:27:03 2022] GET /identity/v3/roles/reader => generated 84 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 01 16:27:04.156871 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c70dfe5-66d9-45bf-a75f-92234bb10e48 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:27:04.188477 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:27:04.220968 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=100 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:27:04.244481 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa134d97-634a-42e3-9113-0c34b881a1ec None admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:27:04.285472 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6636eb9e-5634-4c1b-92e5-9ad5981bf78e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:27:04.286546 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6636eb9e-5634-4c1b-92e5-9ad5981bf78e None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:04.287040 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6636eb9e-5634-4c1b-92e5-9ad5981bf78e None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:04.287617 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6636eb9e-5634-4c1b-92e5-9ad5981bf78e None admin] PATH_INFO: `/v3/roles` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:04.288703 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6636eb9e-5634-4c1b-92e5-9ad5981bf78e None admin] RBAC: Adding query filter params (name=member) {{(pid=80312) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:27:04.289221 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6636eb9e-5634-4c1b-92e5-9ad5981bf78e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:27:04.290020 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c70dfe5-66d9-45bf-a75f-92234bb10e48 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:27:04.291273 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6636eb9e-5634-4c1b-92e5-9ad5981bf78e None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:27:04.300766 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 161/311] 10.222.0.57 () {56 vars in 1101 bytes} [Fri Jul 1 16:27:04 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:27:04.310475 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6636eb9e-5634-4c1b-92e5-9ad5981bf78e None admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:27:04.331737 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1a3c7d-ac76-4612-bc41-a29f4775d569 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:27:04.332365 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1a3c7d-ac76-4612-bc41-a29f4775d569 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:04.332661 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1a3c7d-ac76-4612-bc41-a29f4775d569 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:04.333080 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1a3c7d-ac76-4612-bc41-a29f4775d569 None admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:04.333961 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf1a3c7d-ac76-4612-bc41-a29f4775d569 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:27:04.334324 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf1a3c7d-ac76-4612-bc41-a29f4775d569 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:27:04.335973 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf1a3c7d-ac76-4612-bc41-a29f4775d569 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:27:04.350500 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 151/312] 10.222.0.57 () {56 vars in 1101 bytes} [Fri Jul 1 16:27:04 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 01 16:27:04.352978 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1a3c7d-ac76-4612-bc41-a29f4775d569 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:27:04.468382 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6636eb9e-5634-4c1b-92e5-9ad5981bf78e None admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:27:04.473313 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:27:04.490972 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1a3c7d-ac76-4612-bc41-a29f4775d569 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:27:04.519418 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84873141-d60c-40d2-bf0b-1fd6e5967957 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:27:04.520335 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84873141-d60c-40d2-bf0b-1fd6e5967957 None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:04.520618 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84873141-d60c-40d2-bf0b-1fd6e5967957 None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:04.520898 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84873141-d60c-40d2-bf0b-1fd6e5967957 None admin] PATH_INFO: `/v3/users/6cd5ade2cd254dda8ebb77652bcba049` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:04.522279 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84873141-d60c-40d2-bf0b-1fd6e5967957 None admin] RBAC: Authorizing `identity:get_user(user_id=6cd5ade2cd254dda8ebb77652bcba049)` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:27:04.523878 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84873141-d60c-40d2-bf0b-1fd6e5967957 None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:27:04.526689 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 162/313] 10.222.0.57 () {56 vars in 1177 bytes} [Fri Jul 1 16:27:04 2022] GET /identity/v3/users/6cd5ade2cd254dda8ebb77652bcba049 => generated 293 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 01 16:27:04.531151 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00979909-b611-4c01-b31e-b1eeeab96b76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:27:04.531942 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00979909-b611-4c01-b31e-b1eeeab96b76 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:04.532228 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00979909-b611-4c01-b31e-b1eeeab96b76 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:04.532503 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00979909-b611-4c01-b31e-b1eeeab96b76 None admin] PATH_INFO: `/v3/users/504342654c3b4de58081127b593f0228` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:04.532872 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84873141-d60c-40d2-bf0b-1fd6e5967957 None admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:27:04.535170 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00979909-b611-4c01-b31e-b1eeeab96b76 None admin] RBAC: Authorizing `identity:get_user(user_id=504342654c3b4de58081127b593f0228)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:27:04.536661 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00979909-b611-4c01-b31e-b1eeeab96b76 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:27:04.543338 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 152/314] 10.222.0.57 () {56 vars in 1177 bytes} [Fri Jul 1 16:27:04 2022] GET /identity/v3/users/504342654c3b4de58081127b593f0228 => generated 297 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 01 16:27:04.544844 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00979909-b611-4c01-b31e-b1eeeab96b76 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:27:04.557290 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=100 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:27:04.662050 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00979909-b611-4c01-b31e-b1eeeab96b76 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:27:04.677059 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84873141-d60c-40d2-bf0b-1fd6e5967957 None admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:27:04.707475 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0435f77-9821-46ff-afbb-23a59b25d151 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:27:04.708448 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0435f77-9821-46ff-afbb-23a59b25d151 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:04.708903 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0435f77-9821-46ff-afbb-23a59b25d151 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:04.709362 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0435f77-9821-46ff-afbb-23a59b25d151 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:04.734030 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21d1b6a-f5f5-4d14-b70c-ae1f79ddfe4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:27:04.735513 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:27:04.736328 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21d1b6a-f5f5-4d14-b70c-ae1f79ddfe4a None admin] REQUEST_METHOD: `PUT` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:04.736479 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21d1b6a-f5f5-4d14-b70c-ae1f79ddfe4a None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:04.736778 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21d1b6a-f5f5-4d14-b70c-ae1f79ddfe4a None admin] PATH_INFO: `/v3/system/users/6cd5ade2cd254dda8ebb77652bcba049/roles/983be40bdab04c478eaa90c6b6a46b14` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:04.739620 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c21d1b6a-f5f5-4d14-b70c-ae1f79ddfe4a None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=6cd5ade2cd254dda8ebb77652bcba049, role_id=983be40bdab04c478eaa90c6b6a46b14)` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:27:04.741440 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c21d1b6a-f5f5-4d14-b70c-ae1f79ddfe4a None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:27:04.762250 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 163/315] 10.222.0.57 () {58 vars in 1334 bytes} [Fri Jul 1 16:27:04 2022] PUT /identity/v3/system/users/6cd5ade2cd254dda8ebb77652bcba049/roles/983be40bdab04c478eaa90c6b6a46b14 => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 16:27:04.803909 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0435f77-9821-46ff-afbb-23a59b25d151 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:27:04.807555 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0435f77-9821-46ff-afbb-23a59b25d151 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:27:04.942989 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 153/316] 10.222.0.57 () {58 vars in 1282 bytes} [Fri Jul 1 16:27:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 398 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 01 16:27:04.949161 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21d1b6a-f5f5-4d14-b70c-ae1f79ddfe4a None admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:27:05.040146 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:27:05.068801 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21d1b6a-f5f5-4d14-b70c-ae1f79ddfe4a None admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:27:05.107368 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc077d1-651e-42f2-8279-36ead9ea6ef2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:27:05.108262 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc077d1-651e-42f2-8279-36ead9ea6ef2 None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:05.108547 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc077d1-651e-42f2-8279-36ead9ea6ef2 None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:05.108822 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc077d1-651e-42f2-8279-36ead9ea6ef2 None admin] PATH_INFO: `/v3/projects/63f84f2aa0bf4707bd9551422f0cae3e` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:05.110077 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc077d1-651e-42f2-8279-36ead9ea6ef2 None admin] RBAC: Authorizing `identity:get_project(project_id=63f84f2aa0bf4707bd9551422f0cae3e)` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:27:05.111575 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc077d1-651e-42f2-8279-36ead9ea6ef2 None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:27:05.113616 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 164/317] 10.222.0.57 () {56 vars in 1186 bytes} [Fri Jul 1 16:27:04 2022] GET /identity/v3/projects/63f84f2aa0bf4707bd9551422f0cae3e => generated 302 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:27:05.120531 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0435f77-9821-46ff-afbb-23a59b25d151 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:27:05.182202 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:27:05.240557 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0435f77-9821-46ff-afbb-23a59b25d151 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:27:05.277988 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc8ee08-b0dd-442c-9ae9-e7b5b63fc0dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:27:05.279060 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc8ee08-b0dd-442c-9ae9-e7b5b63fc0dd None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:05.279540 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc8ee08-b0dd-442c-9ae9-e7b5b63fc0dd None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:05.280006 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc8ee08-b0dd-442c-9ae9-e7b5b63fc0dd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:05.281034 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc8ee08-b0dd-442c-9ae9-e7b5b63fc0dd None admin] RBAC: Adding query filter params (role.id=bd7cefce73d14bd7827b58b6bfbe5e7a, scope.project.id=63f84f2aa0bf4707bd9551422f0cae3e, user.id=504342654c3b4de58081127b593f0228) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:27:05.281558 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc8ee08-b0dd-442c-9ae9-e7b5b63fc0dd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:27:05.283247 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc8ee08-b0dd-442c-9ae9-e7b5b63fc0dd None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:27:05.293694 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 154/318] 10.222.0.57 () {56 vars in 1374 bytes} [Fri Jul 1 16:27:05 2022] GET /identity/v3/role_assignments?user.id=504342654c3b4de58081127b593f0228&scope.project.id=63f84f2aa0bf4707bd9551422f0cae3e&role.id=bd7cefce73d14bd7827b58b6bfbe5e7a => generated 608 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:27:05.588587 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d15f6ed-0d89-4082-bec4-f17607616662 None None] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:05.588862 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d15f6ed-0d89-4082-bec4-f17607616662 None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:05.589235 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d15f6ed-0d89-4082-bec4-f17607616662 None None] PATH_INFO: `/` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:05.590743 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 165/319] 10.222.0.57 () {54 vars in 926 bytes} [Fri Jul 1 16:27:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 16:27:05.595778 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4397e16-6f17-439d-a3c1-90c50a0ab5a5 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:05.596067 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4397e16-6f17-439d-a3c1-90c50a0ab5a5 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:05.596349 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4397e16-6f17-439d-a3c1-90c50a0ab5a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:05.614245 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-a4397e16-6f17-439d-a3c1-90c50a0ab5a5 None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:27:05.615479 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-a4397e16-6f17-439d-a3c1-90c50a0ab5a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:27:05.684684 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 155/320] 10.222.0.57 () {58 vars in 1023 bytes} [Fri Jul 1 16:27:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 01 16:27:05.707881 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d15f6ed-0d89-4082-bec4-f17607616662 None None] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:27:05.816164 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d15f6ed-0d89-4082-bec4-f17607616662 None None] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:27:05.863520 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6f97f6-8d3f-48f1-bcd4-16b34e8b3a7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:27:05.864429 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6f97f6-8d3f-48f1-bcd4-16b34e8b3a7f None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:05.864716 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6f97f6-8d3f-48f1-bcd4-16b34e8b3a7f None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:05.864994 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6f97f6-8d3f-48f1-bcd4-16b34e8b3a7f None admin] PATH_INFO: `/v3/domains/default` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:05.866194 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e6f97f6-8d3f-48f1-bcd4-16b34e8b3a7f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:27:05.867673 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e6f97f6-8d3f-48f1-bcd4-16b34e8b3a7f None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:27:05.871153 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 166/321] 10.222.0.57 () {56 vars in 1108 bytes} [Fri Jul 1 16:27:05 2022] GET /identity/v3/domains/default => generated 199 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:27:05.878310 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4397e16-6f17-439d-a3c1-90c50a0ab5a5 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:27:05.929028 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:27:05.988578 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4397e16-6f17-439d-a3c1-90c50a0ab5a5 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:27:06.027580 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1b76ac-38d6-4875-be04-9264c4742382 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:27:06.028608 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1b76ac-38d6-4875-be04-9264c4742382 None admin] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:06.029096 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1b76ac-38d6-4875-be04-9264c4742382 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:06.029573 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1b76ac-38d6-4875-be04-9264c4742382 None admin] PATH_INFO: `/v3/groups` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:06.030524 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f1b76ac-38d6-4875-be04-9264c4742382 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:27:06.032414 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f1b76ac-38d6-4875-be04-9264c4742382 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:27:06.051116 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 156/322] 10.222.0.57 () {60 vars in 1134 bytes} [Fri Jul 1 16:27:05 2022] POST /identity/v3/groups => generated 226 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 16:27:06.100493 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66fd9ee-3050-4d9c-8ec6-701f98c91a75 None None] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:06.101012 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66fd9ee-3050-4d9c-8ec6-701f98c91a75 None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:06.101012 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66fd9ee-3050-4d9c-8ec6-701f98c91a75 None None] PATH_INFO: `/` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:06.102379 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 167/323] 10.222.0.57 () {54 vars in 926 bytes} [Fri Jul 1 16:27:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 16:27:06.108449 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8717334e-6dc0-47d7-90f1-5a4188566bff None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:06.108790 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8717334e-6dc0-47d7-90f1-5a4188566bff None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:06.109016 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8717334e-6dc0-47d7-90f1-5a4188566bff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:06.129242 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-8717334e-6dc0-47d7-90f1-5a4188566bff None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:27:06.130601 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-8717334e-6dc0-47d7-90f1-5a4188566bff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:27:06.207121 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 157/324] 10.222.0.57 () {58 vars in 1023 bytes} [Fri Jul 1 16:27:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 01 16:27:06.229852 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66fd9ee-3050-4d9c-8ec6-701f98c91a75 None None] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:27:06.246723 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=100 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:27:06.344018 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66fd9ee-3050-4d9c-8ec6-701f98c91a75 None None] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:27:06.385364 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af839a6-6ddb-45e0-a7cc-86cd7176991d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:27:06.386095 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af839a6-6ddb-45e0-a7cc-86cd7176991d None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:06.386410 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af839a6-6ddb-45e0-a7cc-86cd7176991d None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:06.386687 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af839a6-6ddb-45e0-a7cc-86cd7176991d None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:06.403377 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af839a6-6ddb-45e0-a7cc-86cd7176991d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 01 16:27:06.411703 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application [None req-5af839a6-6ddb-45e0-a7cc-86cd7176991d None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 01 16:27:06.411703 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 16:27:06.411703 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 16:27:06.411703 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 16:27:06.411703 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 16:27:06.411703 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 16:27:06.411703 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 16:27:06.411703 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 16:27:06.411703 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 16:27:06.411703 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 16:27:06.411703 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 16:27:06.411703 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 16:27:06.411703 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 01 16:27:06.411703 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 01 16:27:06.411703 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 01 16:27:06.411703 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 01 16:27:06.412710 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 01 16:27:06.412710 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 01 16:27:06.412710 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 01 16:27:06.412710 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 01 16:27:06.412710 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 16:27:06.412710 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 16:27:06.412710 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 16:27:06.412710 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 16:27:06.412710 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 16:27:06.412710 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 16:27:06.412710 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 16:27:06.412710 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application with Lock( Jul 01 16:27:06.412710 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 16:27:06.412710 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self._enter() Jul 01 16:27:06.412710 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 16:27:06.412710 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 16:27:06.413521 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 16:27:06.413521 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.creator() Jul 01 16:27:06.413521 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 16:27:06.413521 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application created_value = creator( Jul 01 16:27:06.413521 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 01 16:27:06.413521 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 01 16:27:06.413521 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 01 16:27:06.413521 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 01 16:27:06.413521 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 01 16:27:06.413521 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 01 16:27:06.413521 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 01 16:27:06.413521 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application Jul 01 16:27:06.413521 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 168/325] 10.222.0.57 () {56 vars in 1099 bytes} [Fri Jul 1 16:27:06 2022] GET /identity/v3/roles/reader => generated 84 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 16:27:06.421065 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8717334e-6dc0-47d7-90f1-5a4188566bff None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:27:06.470020 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:27:06.560582 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8717334e-6dc0-47d7-90f1-5a4188566bff None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:27:06.608767 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a102ed03-b1a7-4d4b-937f-ad0ddb4ec8e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:27:06.610441 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a102ed03-b1a7-4d4b-937f-ad0ddb4ec8e0 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:06.611015 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a102ed03-b1a7-4d4b-937f-ad0ddb4ec8e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:06.611490 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a102ed03-b1a7-4d4b-937f-ad0ddb4ec8e0 None admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:06.612469 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a102ed03-b1a7-4d4b-937f-ad0ddb4ec8e0 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:27:06.613000 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a102ed03-b1a7-4d4b-937f-ad0ddb4ec8e0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:27:06.615132 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a102ed03-b1a7-4d4b-937f-ad0ddb4ec8e0 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:27:06.632479 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 158/326] 10.222.0.57 () {56 vars in 1101 bytes} [Fri Jul 1 16:27:06 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 01 16:27:06.638265 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af839a6-6ddb-45e0-a7cc-86cd7176991d None admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:27:06.725735 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=100 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:27:06.763392 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af839a6-6ddb-45e0-a7cc-86cd7176991d None admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:27:06.802335 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea68c5a-9e02-49e8-838a-0fdc521ec078 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:27:06.803307 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea68c5a-9e02-49e8-838a-0fdc521ec078 None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:06.803611 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea68c5a-9e02-49e8-838a-0fdc521ec078 None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:06.803947 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea68c5a-9e02-49e8-838a-0fdc521ec078 None admin] PATH_INFO: `/v3/users/9f0dcfde5e924ebcb328cef131b4bb34` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:06.805467 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea68c5a-9e02-49e8-838a-0fdc521ec078 None admin] RBAC: Authorizing `identity:get_user(user_id=9f0dcfde5e924ebcb328cef131b4bb34)` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:27:06.807179 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea68c5a-9e02-49e8-838a-0fdc521ec078 None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:27:06.809288 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 169/327] 10.222.0.57 () {56 vars in 1177 bytes} [Fri Jul 1 16:27:06 2022] GET /identity/v3/users/9f0dcfde5e924ebcb328cef131b4bb34 => generated 293 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:27:06.814883 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a102ed03-b1a7-4d4b-937f-ad0ddb4ec8e0 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:27:06.960918 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:27:06.966376 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a102ed03-b1a7-4d4b-937f-ad0ddb4ec8e0 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:27:07.008633 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6352df1-c8ab-4897-8dff-e0baf3fb21c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:27:07.009603 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6352df1-c8ab-4897-8dff-e0baf3fb21c2 None admin] REQUEST_METHOD: `PUT` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:07.009877 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6352df1-c8ab-4897-8dff-e0baf3fb21c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:07.010241 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6352df1-c8ab-4897-8dff-e0baf3fb21c2 None admin] PATH_INFO: `/v3/system/users/9f0dcfde5e924ebcb328cef131b4bb34/roles/bd7cefce73d14bd7827b58b6bfbe5e7a` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:07.011878 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6352df1-c8ab-4897-8dff-e0baf3fb21c2 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=9f0dcfde5e924ebcb328cef131b4bb34, role_id=bd7cefce73d14bd7827b58b6bfbe5e7a)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:27:07.013538 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6352df1-c8ab-4897-8dff-e0baf3fb21c2 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:27:07.031475 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 159/328] 10.222.0.57 () {58 vars in 1334 bytes} [Fri Jul 1 16:27:06 2022] PUT /identity/v3/system/users/9f0dcfde5e924ebcb328cef131b4bb34/roles/bd7cefce73d14bd7827b58b6bfbe5e7a => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 16:27:07.088260 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab9d908-e603-41d8-87bc-1528daf57ee8 None None] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:07.088630 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab9d908-e603-41d8-87bc-1528daf57ee8 None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:07.088923 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab9d908-e603-41d8-87bc-1528daf57ee8 None None] PATH_INFO: `/` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:07.090436 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 170/329] 10.222.0.57 () {54 vars in 926 bytes} [Fri Jul 1 16:27:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 01 16:27:07.096751 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb2ab49-f3ff-409c-bc14-edbff3a2681a None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:07.097050 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb2ab49-f3ff-409c-bc14-edbff3a2681a None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:07.097303 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb2ab49-f3ff-409c-bc14-edbff3a2681a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:07.122375 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-5eb2ab49-f3ff-409c-bc14-edbff3a2681a None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:27:07.124444 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-5eb2ab49-f3ff-409c-bc14-edbff3a2681a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:27:07.216849 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 160/330] 10.222.0.57 () {58 vars in 1023 bytes} [Fri Jul 1 16:27:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 01 16:27:07.242263 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab9d908-e603-41d8-87bc-1528daf57ee8 None None] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:27:07.382153 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab9d908-e603-41d8-87bc-1528daf57ee8 None None] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:27:07.426688 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3850bc-28e4-4006-9a1a-a88f534a2430 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:27:07.427506 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3850bc-28e4-4006-9a1a-a88f534a2430 None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:07.427790 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3850bc-28e4-4006-9a1a-a88f534a2430 None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:07.428059 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3850bc-28e4-4006-9a1a-a88f534a2430 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:07.439698 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=100 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:27:07.444954 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a3850bc-28e4-4006-9a1a-a88f534a2430 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 01 16:27:07.454690 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application [None req-5a3850bc-28e4-4006-9a1a-a88f534a2430 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 01 16:27:07.454690 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 16:27:07.454690 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 16:27:07.454690 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 16:27:07.454690 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 16:27:07.454690 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 16:27:07.454690 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 16:27:07.454690 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 16:27:07.454690 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 16:27:07.454690 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 16:27:07.454690 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 16:27:07.454690 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 16:27:07.454690 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 01 16:27:07.454690 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 01 16:27:07.454690 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 01 16:27:07.454690 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 01 16:27:07.455844 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 01 16:27:07.455844 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 01 16:27:07.455844 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 01 16:27:07.455844 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 01 16:27:07.455844 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 16:27:07.455844 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 16:27:07.455844 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 16:27:07.455844 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 16:27:07.455844 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 16:27:07.455844 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 16:27:07.455844 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 16:27:07.455844 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application with Lock( Jul 01 16:27:07.455844 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 16:27:07.455844 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self._enter() Jul 01 16:27:07.455844 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 16:27:07.455844 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 16:27:07.456674 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 16:27:07.456674 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.creator() Jul 01 16:27:07.456674 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 16:27:07.456674 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application created_value = creator( Jul 01 16:27:07.456674 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 01 16:27:07.456674 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 01 16:27:07.456674 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 01 16:27:07.456674 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 01 16:27:07.456674 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 01 16:27:07.456674 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 01 16:27:07.456674 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 01 16:27:07.456674 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application Jul 01 16:27:07.456674 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 171/331] 10.222.0.57 () {56 vars in 1099 bytes} [Fri Jul 1 16:27:07 2022] GET /identity/v3/roles/member => generated 84 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 16:27:07.462363 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb2ab49-f3ff-409c-bc14-edbff3a2681a None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:27:07.567840 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:27:07.580726 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb2ab49-f3ff-409c-bc14-edbff3a2681a None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:27:07.629682 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc20e8f2-8e9c-4fa4-8aad-dbd14f940fb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:27:07.630744 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc20e8f2-8e9c-4fa4-8aad-dbd14f940fb4 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:07.631229 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc20e8f2-8e9c-4fa4-8aad-dbd14f940fb4 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:07.631702 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc20e8f2-8e9c-4fa4-8aad-dbd14f940fb4 None admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:07.632683 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc20e8f2-8e9c-4fa4-8aad-dbd14f940fb4 None admin] RBAC: Adding query filter params (name=member) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:27:07.633191 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc20e8f2-8e9c-4fa4-8aad-dbd14f940fb4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:27:07.635546 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc20e8f2-8e9c-4fa4-8aad-dbd14f940fb4 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:27:07.650941 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 161/332] 10.222.0.57 () {56 vars in 1101 bytes} [Fri Jul 1 16:27:07 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:27:07.656160 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3850bc-28e4-4006-9a1a-a88f534a2430 None admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:27:07.794840 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3850bc-28e4-4006-9a1a-a88f534a2430 None admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:27:07.847286 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeea04a0-3ed8-453f-9a68-b4167d445edf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:27:07.848198 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeea04a0-3ed8-453f-9a68-b4167d445edf None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:07.848554 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeea04a0-3ed8-453f-9a68-b4167d445edf None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:07.848843 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeea04a0-3ed8-453f-9a68-b4167d445edf None admin] PATH_INFO: `/v3/users/6cd5ade2cd254dda8ebb77652bcba049` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:07.851167 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeea04a0-3ed8-453f-9a68-b4167d445edf None admin] RBAC: Authorizing `identity:get_user(user_id=6cd5ade2cd254dda8ebb77652bcba049)` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:27:07.853356 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeea04a0-3ed8-453f-9a68-b4167d445edf None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:27:07.856017 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 172/333] 10.222.0.57 () {56 vars in 1177 bytes} [Fri Jul 1 16:27:07 2022] GET /identity/v3/users/6cd5ade2cd254dda8ebb77652bcba049 => generated 293 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:27:07.861248 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc20e8f2-8e9c-4fa4-8aad-dbd14f940fb4 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:27:07.985674 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc20e8f2-8e9c-4fa4-8aad-dbd14f940fb4 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:27:08.020929 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc175ff-4884-4036-8ef6-bfdab5a96789 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:27:08.021589 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc175ff-4884-4036-8ef6-bfdab5a96789 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:08.021871 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc175ff-4884-4036-8ef6-bfdab5a96789 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:08.022197 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc175ff-4884-4036-8ef6-bfdab5a96789 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:08.023076 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc175ff-4884-4036-8ef6-bfdab5a96789 None admin] RBAC: Adding query filter params (role.id=983be40bdab04c478eaa90c6b6a46b14, user.id=6cd5ade2cd254dda8ebb77652bcba049, scope.system=all) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:27:08.023386 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc175ff-4884-4036-8ef6-bfdab5a96789 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:27:08.024939 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc175ff-4884-4036-8ef6-bfdab5a96789 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:27:08.037826 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 162/334] 10.222.0.57 () {56 vars in 1308 bytes} [Fri Jul 1 16:27:07 2022] GET /identity/v3/role_assignments?user.id=6cd5ade2cd254dda8ebb77652bcba049&scope.system=all&role.id=983be40bdab04c478eaa90c6b6a46b14 => generated 510 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:27:08.662063 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4a6e4d-e187-489a-9caf-8b3b441623fa None None] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:08.662885 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4a6e4d-e187-489a-9caf-8b3b441623fa None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:08.662885 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4a6e4d-e187-489a-9caf-8b3b441623fa None None] PATH_INFO: `/` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:08.664324 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 173/335] 10.222.0.57 () {54 vars in 926 bytes} [Fri Jul 1 16:27:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 16:27:08.670017 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0d73bf-dcdb-473a-8e1b-ccc4552cad32 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:08.670315 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0d73bf-dcdb-473a-8e1b-ccc4552cad32 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:08.670625 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0d73bf-dcdb-473a-8e1b-ccc4552cad32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:08.690386 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-4e0d73bf-dcdb-473a-8e1b-ccc4552cad32 None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:27:08.691893 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-4e0d73bf-dcdb-473a-8e1b-ccc4552cad32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:27:08.700970 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:27:08.781424 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 163/336] 10.222.0.57 () {58 vars in 1023 bytes} [Fri Jul 1 16:27:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 01 16:27:08.824150 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4a6e4d-e187-489a-9caf-8b3b441623fa None None] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:27:08.906655 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=100 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:27:08.945217 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4a6e4d-e187-489a-9caf-8b3b441623fa None None] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:27:08.983095 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d1f742-5e42-44b2-a2a2-267055157eee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:27:08.983894 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d1f742-5e42-44b2-a2a2-267055157eee None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:08.984176 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d1f742-5e42-44b2-a2a2-267055157eee None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:08.984451 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d1f742-5e42-44b2-a2a2-267055157eee None admin] PATH_INFO: `/v3/roles/member` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:08.994799 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d1f742-5e42-44b2-a2a2-267055157eee None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 01 16:27:09.010246 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application [None req-e6d1f742-5e42-44b2-a2a2-267055157eee None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 01 16:27:09.010246 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 16:27:09.010246 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 16:27:09.010246 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 16:27:09.010246 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 16:27:09.010246 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 16:27:09.010246 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 16:27:09.010246 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 16:27:09.010246 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 16:27:09.010246 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 16:27:09.010246 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 16:27:09.010246 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 16:27:09.010246 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 01 16:27:09.010246 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 01 16:27:09.010246 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 01 16:27:09.010246 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 01 16:27:09.011392 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 01 16:27:09.011392 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 01 16:27:09.011392 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 01 16:27:09.011392 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 01 16:27:09.011392 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 16:27:09.011392 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 16:27:09.011392 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 16:27:09.011392 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 16:27:09.011392 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 16:27:09.011392 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 16:27:09.011392 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 16:27:09.011392 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application with Lock( Jul 01 16:27:09.011392 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 16:27:09.011392 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self._enter() Jul 01 16:27:09.011392 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 16:27:09.011392 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 16:27:09.012226 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 16:27:09.012226 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.creator() Jul 01 16:27:09.012226 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 16:27:09.012226 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application created_value = creator( Jul 01 16:27:09.012226 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 01 16:27:09.012226 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 01 16:27:09.012226 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 01 16:27:09.012226 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 01 16:27:09.012226 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 01 16:27:09.012226 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 01 16:27:09.012226 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 01 16:27:09.012226 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application Jul 01 16:27:09.012226 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 174/337] 10.222.0.57 () {56 vars in 1099 bytes} [Fri Jul 1 16:27:08 2022] GET /identity/v3/roles/member => generated 84 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 16:27:09.016417 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0d73bf-dcdb-473a-8e1b-ccc4552cad32 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:27:09.153650 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0d73bf-dcdb-473a-8e1b-ccc4552cad32 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:27:09.217549 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858ce9a4-71c1-4794-a47f-24263c081cbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:27:09.218203 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858ce9a4-71c1-4794-a47f-24263c081cbf None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:09.218521 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858ce9a4-71c1-4794-a47f-24263c081cbf None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:09.218794 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858ce9a4-71c1-4794-a47f-24263c081cbf None admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:09.219559 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-858ce9a4-71c1-4794-a47f-24263c081cbf None admin] RBAC: Adding query filter params (name=member) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:27:09.219862 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-858ce9a4-71c1-4794-a47f-24263c081cbf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:27:09.221327 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-858ce9a4-71c1-4794-a47f-24263c081cbf None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:27:09.231042 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 164/338] 10.222.0.57 () {56 vars in 1101 bytes} [Fri Jul 1 16:27:09 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:27:09.239458 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d1f742-5e42-44b2-a2a2-267055157eee None admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:27:09.447532 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d1f742-5e42-44b2-a2a2-267055157eee None admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:27:09.486673 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=100 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:27:09.502411 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dad0f09-34e8-483d-b9b7-e863c97accd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:27:09.503447 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dad0f09-34e8-483d-b9b7-e863c97accd9 None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:09.504163 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dad0f09-34e8-483d-b9b7-e863c97accd9 None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:09.504756 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dad0f09-34e8-483d-b9b7-e863c97accd9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:09.600186 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dad0f09-34e8-483d-b9b7-e863c97accd9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:27:09.602024 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dad0f09-34e8-483d-b9b7-e863c97accd9 None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:27:09.728737 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 175/339] 10.222.0.57 () {58 vars in 1282 bytes} [Fri Jul 1 16:27:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 490 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 01 16:27:09.734033 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858ce9a4-71c1-4794-a47f-24263c081cbf None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:27:09.745168 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:27:09.885797 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a73a848-bbdc-49c8-8529-8906e50bb0e8 None None] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:09.886122 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a73a848-bbdc-49c8-8529-8906e50bb0e8 None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:09.886406 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a73a848-bbdc-49c8-8529-8906e50bb0e8 None None] PATH_INFO: `/` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:09.888478 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 176/340] 10.222.0.57 () {54 vars in 926 bytes} [Fri Jul 1 16:27:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 16:27:09.894846 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54e2118-06a3-4126-a556-69ce8aae1fc3 None None] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:09.895421 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54e2118-06a3-4126-a556-69ce8aae1fc3 None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:09.895896 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54e2118-06a3-4126-a556-69ce8aae1fc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:09.913409 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858ce9a4-71c1-4794-a47f-24263c081cbf None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:27:09.928247 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.auth.core [None req-f54e2118-06a3-4126-a556-69ce8aae1fc3 None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80312) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:27:09.929765 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-f54e2118-06a3-4126-a556-69ce8aae1fc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:27:09.982784 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=100 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:27:09.986729 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4427b9-1be7-42e2-9d3f-1797b9f8841c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:27:09.987750 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4427b9-1be7-42e2-9d3f-1797b9f8841c None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:09.988216 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4427b9-1be7-42e2-9d3f-1797b9f8841c None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:09.988911 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4427b9-1be7-42e2-9d3f-1797b9f8841c None admin] PATH_INFO: `/v3/projects/5bef204363a04d5d8ceab527b8803f6e` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:09.991075 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab4427b9-1be7-42e2-9d3f-1797b9f8841c None admin] RBAC: Authorizing `identity:get_project(project_id=5bef204363a04d5d8ceab527b8803f6e)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:27:09.993039 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab4427b9-1be7-42e2-9d3f-1797b9f8841c None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:27:09.996134 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 165/341] 10.222.0.57 () {56 vars in 1186 bytes} [Fri Jul 1 16:27:09 2022] GET /identity/v3/projects/5bef204363a04d5d8ceab527b8803f6e => generated 298 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:27:10.002962 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4427b9-1be7-42e2-9d3f-1797b9f8841c None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:27:10.086958 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 177/342] 10.222.0.57 () {58 vars in 1023 bytes} [Fri Jul 1 16:27:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 01 16:27:10.108469 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b507f575-a676-4591-b68e-c0cf7431ccad None None] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:10.108770 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b507f575-a676-4591-b68e-c0cf7431ccad None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:10.109037 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b507f575-a676-4591-b68e-c0cf7431ccad None None] PATH_INFO: `/` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:10.110661 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 178/343] 10.222.0.57 () {54 vars in 926 bytes} [Fri Jul 1 16:27:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 01 16:27:10.117387 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528172c9-8ba6-4b87-bf4c-d3ccd3c63406 None None] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:10.117676 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528172c9-8ba6-4b87-bf4c-d3ccd3c63406 None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:10.117951 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528172c9-8ba6-4b87-bf4c-d3ccd3c63406 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:10.147498 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.auth.core [None req-528172c9-8ba6-4b87-bf4c-d3ccd3c63406 None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80312) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:27:10.148970 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-528172c9-8ba6-4b87-bf4c-d3ccd3c63406 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:27:10.168425 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:27:10.230858 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4427b9-1be7-42e2-9d3f-1797b9f8841c None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:27:10.297877 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 179/344] 10.222.0.57 () {58 vars in 1023 bytes} [Fri Jul 1 16:27:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 01 16:27:10.298842 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528172c9-8ba6-4b87-bf4c-d3ccd3c63406 None None] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:27:10.338446 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9c86c3-14d9-4031-851b-c13ef1d75347 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:27:10.339323 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9c86c3-14d9-4031-851b-c13ef1d75347 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:10.339619 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9c86c3-14d9-4031-851b-c13ef1d75347 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:10.339902 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9c86c3-14d9-4031-851b-c13ef1d75347 None admin] PATH_INFO: `/v3/groups/eb554eeac000489bb0d030619756ec4b` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:10.355415 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd9c86c3-14d9-4031-851b-c13ef1d75347 None admin] RBAC: Authorizing `identity:get_group(group_id=eb554eeac000489bb0d030619756ec4b)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:27:10.357195 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd9c86c3-14d9-4031-851b-c13ef1d75347 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:27:10.359025 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 166/345] 10.222.0.57 () {56 vars in 1180 bytes} [Fri Jul 1 16:27:10 2022] GET /identity/v3/groups/eb554eeac000489bb0d030619756ec4b => generated 226 bytes in 357 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:27:10.360236 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9c86c3-14d9-4031-851b-c13ef1d75347 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:27:10.498760 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=100 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:27:10.504529 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9c86c3-14d9-4031-851b-c13ef1d75347 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:27:10.533633 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528172c9-8ba6-4b87-bf4c-d3ccd3c63406 None None] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:27:10.563926 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a190a66e-b98c-4a85-8b25-6a265f4cf7ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:27:10.564660 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a190a66e-b98c-4a85-8b25-6a265f4cf7ba None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:10.564949 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a190a66e-b98c-4a85-8b25-6a265f4cf7ba None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:10.565323 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a190a66e-b98c-4a85-8b25-6a265f4cf7ba None admin] PATH_INFO: `/v3/roles/member` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:10.574733 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:27:10.593582 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a190a66e-b98c-4a85-8b25-6a265f4cf7ba None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 01 16:27:10.601625 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application [None req-a190a66e-b98c-4a85-8b25-6a265f4cf7ba None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 01 16:27:10.601625 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 16:27:10.601625 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 16:27:10.601625 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 16:27:10.601625 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 16:27:10.601625 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 16:27:10.601625 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 16:27:10.601625 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 16:27:10.601625 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 16:27:10.601625 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 16:27:10.601625 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 16:27:10.601625 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 16:27:10.601625 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 01 16:27:10.601625 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 01 16:27:10.601625 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 01 16:27:10.601625 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 01 16:27:10.602581 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 01 16:27:10.602581 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 01 16:27:10.602581 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 01 16:27:10.602581 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 01 16:27:10.602581 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 16:27:10.602581 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 16:27:10.602581 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 16:27:10.602581 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 16:27:10.602581 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 16:27:10.602581 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 16:27:10.602581 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 16:27:10.602581 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application with Lock( Jul 01 16:27:10.602581 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 16:27:10.602581 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._enter() Jul 01 16:27:10.602581 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 16:27:10.602581 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 16:27:10.603552 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 16:27:10.603552 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.creator() Jul 01 16:27:10.603552 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 16:27:10.603552 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application created_value = creator( Jul 01 16:27:10.603552 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 01 16:27:10.603552 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 01 16:27:10.603552 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 01 16:27:10.603552 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 01 16:27:10.603552 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 01 16:27:10.603552 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 01 16:27:10.603552 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 01 16:27:10.603552 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Jul 01 16:27:10.607197 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 167/346] 10.222.0.57 () {56 vars in 1099 bytes} [Fri Jul 1 16:27:10 2022] GET /identity/v3/roles/member => generated 84 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 16:27:10.608319 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a190a66e-b98c-4a85-8b25-6a265f4cf7ba None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:27:10.616782 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f055140-5c32-4a05-8b88-ecde324af7bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:27:10.617729 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f055140-5c32-4a05-8b88-ecde324af7bf None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:10.618021 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f055140-5c32-4a05-8b88-ecde324af7bf None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:10.618851 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f055140-5c32-4a05-8b88-ecde324af7bf None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:10.647673 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f055140-5c32-4a05-8b88-ecde324af7bf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 01 16:27:10.656881 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application [None req-0f055140-5c32-4a05-8b88-ecde324af7bf None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 01 16:27:10.656881 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 16:27:10.656881 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 16:27:10.656881 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 16:27:10.656881 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 16:27:10.656881 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 16:27:10.656881 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 16:27:10.656881 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 16:27:10.656881 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 16:27:10.656881 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 16:27:10.656881 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 16:27:10.656881 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 16:27:10.656881 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 01 16:27:10.656881 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 01 16:27:10.656881 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 01 16:27:10.656881 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 01 16:27:10.657952 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 01 16:27:10.657952 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 01 16:27:10.657952 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 01 16:27:10.657952 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 01 16:27:10.657952 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 16:27:10.657952 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 16:27:10.657952 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 16:27:10.657952 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 16:27:10.657952 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 16:27:10.657952 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 16:27:10.657952 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 16:27:10.657952 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application with Lock( Jul 01 16:27:10.657952 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 16:27:10.657952 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self._enter() Jul 01 16:27:10.657952 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 16:27:10.657952 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 16:27:10.658815 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 16:27:10.658815 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.creator() Jul 01 16:27:10.658815 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 16:27:10.658815 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application created_value = creator( Jul 01 16:27:10.658815 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 01 16:27:10.658815 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 01 16:27:10.658815 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 01 16:27:10.658815 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 01 16:27:10.658815 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 01 16:27:10.658815 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 01 16:27:10.658815 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 01 16:27:10.658815 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application Jul 01 16:27:10.662933 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 180/347] 10.222.0.57 () {56 vars in 1114 bytes} [Fri Jul 1 16:27:10 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 360 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 16:27:10.664398 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f055140-5c32-4a05-8b88-ecde324af7bf None admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:27:10.794884 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f055140-5c32-4a05-8b88-ecde324af7bf None admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:27:10.806761 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a190a66e-b98c-4a85-8b25-6a265f4cf7ba None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:27:10.838280 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62631fe2-cbb5-4394-8ee1-2d464ddbe9c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:27:10.839255 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62631fe2-cbb5-4394-8ee1-2d464ddbe9c9 None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:10.839709 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62631fe2-cbb5-4394-8ee1-2d464ddbe9c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:10.840140 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62631fe2-cbb5-4394-8ee1-2d464ddbe9c9 None admin] PATH_INFO: `/v3/roles` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:10.841071 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62631fe2-cbb5-4394-8ee1-2d464ddbe9c9 None admin] RBAC: Adding query filter params (name=member) {{(pid=80312) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:27:10.841541 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62631fe2-cbb5-4394-8ee1-2d464ddbe9c9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:27:10.843635 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62631fe2-cbb5-4394-8ee1-2d464ddbe9c9 None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:27:10.857305 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 181/348] 10.222.0.57 () {56 vars in 1101 bytes} [Fri Jul 1 16:27:10 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:27:10.859500 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62631fe2-cbb5-4394-8ee1-2d464ddbe9c9 None admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:27:10.883633 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46d45e9-3334-4abe-a9d3-e737c0b21064 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:27:10.884332 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46d45e9-3334-4abe-a9d3-e737c0b21064 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:10.884664 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46d45e9-3334-4abe-a9d3-e737c0b21064 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:10.884941 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46d45e9-3334-4abe-a9d3-e737c0b21064 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:10.885990 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d46d45e9-3334-4abe-a9d3-e737c0b21064 None admin] RBAC: Adding query filter params (group.id=eb554eeac000489bb0d030619756ec4b, role.id=983be40bdab04c478eaa90c6b6a46b14, scope.project.id=5bef204363a04d5d8ceab527b8803f6e) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:27:10.886307 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d46d45e9-3334-4abe-a9d3-e737c0b21064 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:27:10.889441 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=100 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:27:10.890588 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d46d45e9-3334-4abe-a9d3-e737c0b21064 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:27:10.900165 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 168/349] 10.222.0.57 () {56 vars in 1376 bytes} [Fri Jul 1 16:27:10 2022] GET /identity/v3/role_assignments?group.id=eb554eeac000489bb0d030619756ec4b&scope.project.id=5bef204363a04d5d8ceab527b8803f6e&role.id=983be40bdab04c478eaa90c6b6a46b14 => generated 260 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:27:10.901954 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4bfb39-42f7-4113-b06e-f7a5508e5d40 None None] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:10.902256 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4bfb39-42f7-4113-b06e-f7a5508e5d40 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:10.906710 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4bfb39-42f7-4113-b06e-f7a5508e5d40 None None] PATH_INFO: `/` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:10.911410 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 169/350] 10.222.0.57 () {54 vars in 926 bytes} [Fri Jul 1 16:27:10 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 16:27:10.912598 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4bfb39-42f7-4113-b06e-f7a5508e5d40 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:27:11.016288 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:27:11.022712 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62631fe2-cbb5-4394-8ee1-2d464ddbe9c9 None admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:27:11.066017 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ed75b6-153c-4621-ab16-d7d4ea706091 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:27:11.067176 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ed75b6-153c-4621-ab16-d7d4ea706091 None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:11.067767 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ed75b6-153c-4621-ab16-d7d4ea706091 None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:11.068238 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ed75b6-153c-4621-ab16-d7d4ea706091 None admin] PATH_INFO: `/v3/roles` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:11.069269 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ed75b6-153c-4621-ab16-d7d4ea706091 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80312) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:27:11.069778 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ed75b6-153c-4621-ab16-d7d4ea706091 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:27:11.072216 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ed75b6-153c-4621-ab16-d7d4ea706091 None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:27:11.081789 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4bfb39-42f7-4113-b06e-f7a5508e5d40 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:27:11.083688 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 182/351] 10.222.0.57 () {56 vars in 1111 bytes} [Fri Jul 1 16:27:10 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:27:11.086334 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80069c1a-3d68-4041-a29e-e8e08b7f277a None None] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:11.086864 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80069c1a-3d68-4041-a29e-e8e08b7f277a None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:11.087399 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80069c1a-3d68-4041-a29e-e8e08b7f277a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:11.108587 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.auth.core [None req-80069c1a-3d68-4041-a29e-e8e08b7f277a None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80312) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:27:11.110672 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-80069c1a-3d68-4041-a29e-e8e08b7f277a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:27:11.124714 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056da442-cf68-438e-b55d-cc50a07ddb48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:27:11.125379 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056da442-cf68-438e-b55d-cc50a07ddb48 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:11.125668 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056da442-cf68-438e-b55d-cc50a07ddb48 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:11.125947 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056da442-cf68-438e-b55d-cc50a07ddb48 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:11.193595 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 183/352] 10.222.0.57 () {58 vars in 1023 bytes} [Fri Jul 1 16:27:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 01 16:27:11.195859 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1581f96b-55c0-4870-a951-a2245cea801a None None] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:11.196329 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1581f96b-55c0-4870-a951-a2245cea801a None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:11.196827 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1581f96b-55c0-4870-a951-a2245cea801a None None] PATH_INFO: `/` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:11.198642 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 184/353] 10.222.0.57 () {54 vars in 926 bytes} [Fri Jul 1 16:27:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 16:27:11.199632 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1581f96b-55c0-4870-a951-a2245cea801a None None] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:27:11.206956 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=100 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:27:11.227198 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-056da442-cf68-438e-b55d-cc50a07ddb48 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:27:11.228983 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-056da442-cf68-438e-b55d-cc50a07ddb48 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:27:11.305050 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:27:11.358533 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1581f96b-55c0-4870-a951-a2245cea801a None None] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:27:11.364702 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 170/354] 10.222.0.57 () {58 vars in 1282 bytes} [Fri Jul 1 16:27:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 452 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 01 16:27:11.366289 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7111184f-c85f-44d0-994b-e2a85070c3ef None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:11.366838 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7111184f-c85f-44d0-994b-e2a85070c3ef None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:11.367333 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7111184f-c85f-44d0-994b-e2a85070c3ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:11.387349 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-7111184f-c85f-44d0-994b-e2a85070c3ef None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:27:11.388576 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-7111184f-c85f-44d0-994b-e2a85070c3ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:27:11.405272 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532350b9-35bc-4ec0-8b84-82d45bda4842 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:27:11.405937 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532350b9-35bc-4ec0-8b84-82d45bda4842 None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:11.406219 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532350b9-35bc-4ec0-8b84-82d45bda4842 None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:11.406584 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532350b9-35bc-4ec0-8b84-82d45bda4842 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:11.483455 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 171/355] 10.222.0.57 () {58 vars in 1023 bytes} [Fri Jul 1 16:27:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 01 16:27:11.484586 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7111184f-c85f-44d0-994b-e2a85070c3ef None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:27:11.501562 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-532350b9-35bc-4ec0-8b84-82d45bda4842 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:27:11.503548 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-532350b9-35bc-4ec0-8b84-82d45bda4842 None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:27:11.507096 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=100 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:27:11.606133 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:27:11.621251 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7111184f-c85f-44d0-994b-e2a85070c3ef None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:27:11.639002 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 185/356] 10.222.0.57 () {58 vars in 1282 bytes} [Fri Jul 1 16:27:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 440 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 01 16:27:11.640415 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532350b9-35bc-4ec0-8b84-82d45bda4842 None admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:27:11.663955 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c0a445-d386-4bd9-82f2-854c78aa67de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:27:11.664768 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c0a445-d386-4bd9-82f2-854c78aa67de None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:11.665048 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c0a445-d386-4bd9-82f2-854c78aa67de None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:11.665888 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c0a445-d386-4bd9-82f2-854c78aa67de None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:11.679585 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c0a445-d386-4bd9-82f2-854c78aa67de None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 01 16:27:11.688247 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application [None req-a9c0a445-d386-4bd9-82f2-854c78aa67de None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 01 16:27:11.688247 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 16:27:11.688247 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 16:27:11.688247 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 16:27:11.688247 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 16:27:11.688247 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 16:27:11.688247 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 16:27:11.688247 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 16:27:11.688247 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 16:27:11.688247 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 16:27:11.688247 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 16:27:11.688247 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 16:27:11.688247 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 01 16:27:11.688247 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 01 16:27:11.688247 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 01 16:27:11.688247 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 01 16:27:11.689283 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 01 16:27:11.689283 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 01 16:27:11.689283 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 01 16:27:11.689283 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 01 16:27:11.689283 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 16:27:11.689283 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 16:27:11.689283 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 16:27:11.689283 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 16:27:11.689283 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 16:27:11.689283 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 16:27:11.689283 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 16:27:11.689283 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application with Lock( Jul 01 16:27:11.689283 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 16:27:11.689283 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._enter() Jul 01 16:27:11.689283 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 16:27:11.689283 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 16:27:11.691234 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 16:27:11.691234 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.creator() Jul 01 16:27:11.691234 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 16:27:11.691234 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application created_value = creator( Jul 01 16:27:11.691234 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 01 16:27:11.691234 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 01 16:27:11.691234 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 01 16:27:11.691234 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 01 16:27:11.691234 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 01 16:27:11.691234 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 01 16:27:11.691234 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 01 16:27:11.691234 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Jul 01 16:27:11.691234 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 172/357] 10.222.0.57 () {56 vars in 1114 bytes} [Fri Jul 1 16:27:11 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 16:27:11.691234 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c7f73b-7bb2-412f-8de9-deeeac3d31c3 None None] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:11.692240 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c7f73b-7bb2-412f-8de9-deeeac3d31c3 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:11.692240 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c7f73b-7bb2-412f-8de9-deeeac3d31c3 None None] PATH_INFO: `/` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:11.693778 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 173/358] 10.222.0.57 () {54 vars in 926 bytes} [Fri Jul 1 16:27:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 16:27:11.693778 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c7f73b-7bb2-412f-8de9-deeeac3d31c3 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:27:11.783245 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532350b9-35bc-4ec0-8b84-82d45bda4842 None admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:27:11.784816 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=100 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:27:11.834413 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c58f29-e7bd-4dae-a3e1-b95e0507936d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:27:11.835363 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c58f29-e7bd-4dae-a3e1-b95e0507936d None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:11.835648 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c58f29-e7bd-4dae-a3e1-b95e0507936d None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:11.835924 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c58f29-e7bd-4dae-a3e1-b95e0507936d None admin] PATH_INFO: `/v3/projects/63f84f2aa0bf4707bd9551422f0cae3e` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:11.837700 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c58f29-e7bd-4dae-a3e1-b95e0507936d None admin] RBAC: Authorizing `identity:get_project(project_id=63f84f2aa0bf4707bd9551422f0cae3e)` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:27:11.839451 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c58f29-e7bd-4dae-a3e1-b95e0507936d None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:27:11.841341 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 186/359] 10.222.0.57 () {56 vars in 1186 bytes} [Fri Jul 1 16:27:11 2022] GET /identity/v3/projects/63f84f2aa0bf4707bd9551422f0cae3e => generated 302 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:27:11.842085 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c58f29-e7bd-4dae-a3e1-b95e0507936d None admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:27:11.857846 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c7f73b-7bb2-412f-8de9-deeeac3d31c3 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:27:11.902950 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5daa2f-8169-4290-86ad-0aba7b4050bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:27:11.903691 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5daa2f-8169-4290-86ad-0aba7b4050bb None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:11.904000 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5daa2f-8169-4290-86ad-0aba7b4050bb None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:11.904341 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5daa2f-8169-4290-86ad-0aba7b4050bb None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:11.917323 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d5daa2f-8169-4290-86ad-0aba7b4050bb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 01 16:27:11.927830 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application [None req-9d5daa2f-8169-4290-86ad-0aba7b4050bb None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 01 16:27:11.927830 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 16:27:11.927830 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 16:27:11.927830 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 16:27:11.927830 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 16:27:11.927830 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 16:27:11.927830 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 16:27:11.927830 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 16:27:11.927830 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 16:27:11.927830 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 16:27:11.927830 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 16:27:11.927830 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 16:27:11.927830 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 01 16:27:11.927830 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 01 16:27:11.927830 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 01 16:27:11.927830 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 01 16:27:11.928894 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 01 16:27:11.928894 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 01 16:27:11.928894 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 01 16:27:11.928894 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 01 16:27:11.928894 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 16:27:11.928894 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 16:27:11.928894 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 16:27:11.928894 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 16:27:11.928894 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 16:27:11.928894 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 16:27:11.928894 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 16:27:11.928894 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application with Lock( Jul 01 16:27:11.928894 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 16:27:11.928894 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._enter() Jul 01 16:27:11.928894 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 16:27:11.928894 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 16:27:11.929708 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 16:27:11.929708 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.creator() Jul 01 16:27:11.929708 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 16:27:11.929708 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application created_value = creator( Jul 01 16:27:11.929708 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 01 16:27:11.929708 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 01 16:27:11.929708 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 01 16:27:11.929708 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 01 16:27:11.929708 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 01 16:27:11.929708 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 01 16:27:11.929708 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 01 16:27:11.929708 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Jul 01 16:27:11.934425 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 174/360] 10.222.0.57 () {56 vars in 1099 bytes} [Fri Jul 1 16:27:11 2022] GET /identity/v3/roles/reader => generated 84 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 16:27:11.935735 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5daa2f-8169-4290-86ad-0aba7b4050bb None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:27:11.949919 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:27:11.980955 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c58f29-e7bd-4dae-a3e1-b95e0507936d None admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:27:12.027983 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475255fa-1240-4c2a-a0e0-fed54aa57592 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:27:12.028857 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475255fa-1240-4c2a-a0e0-fed54aa57592 None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:12.029141 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475255fa-1240-4c2a-a0e0-fed54aa57592 None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:12.029420 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475255fa-1240-4c2a-a0e0-fed54aa57592 None admin] PATH_INFO: `/v3/projects/5bef204363a04d5d8ceab527b8803f6e` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:12.032039 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-475255fa-1240-4c2a-a0e0-fed54aa57592 None admin] RBAC: Authorizing `identity:get_project(project_id=5bef204363a04d5d8ceab527b8803f6e)` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:27:12.032169 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-475255fa-1240-4c2a-a0e0-fed54aa57592 None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:27:12.035356 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 187/361] 10.222.0.57 () {56 vars in 1186 bytes} [Fri Jul 1 16:27:11 2022] GET /identity/v3/projects/5bef204363a04d5d8ceab527b8803f6e => generated 298 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:27:12.035356 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e85ee37-09f4-4759-9fd0-6144f81ddc58 None None] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:12.035356 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e85ee37-09f4-4759-9fd0-6144f81ddc58 None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:12.035614 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e85ee37-09f4-4759-9fd0-6144f81ddc58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:12.061237 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.auth.core [None req-8e85ee37-09f4-4759-9fd0-6144f81ddc58 None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80312) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:27:12.063589 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-8e85ee37-09f4-4759-9fd0-6144f81ddc58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:27:12.095693 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5daa2f-8169-4290-86ad-0aba7b4050bb None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:27:12.109143 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=100 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:27:12.146966 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70a1987-5b32-4c96-9a76-ddf1a7dd8ae3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:27:12.147639 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70a1987-5b32-4c96-9a76-ddf1a7dd8ae3 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:12.147927 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70a1987-5b32-4c96-9a76-ddf1a7dd8ae3 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:12.148206 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70a1987-5b32-4c96-9a76-ddf1a7dd8ae3 None admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:12.148970 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b70a1987-5b32-4c96-9a76-ddf1a7dd8ae3 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:27:12.149282 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b70a1987-5b32-4c96-9a76-ddf1a7dd8ae3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:27:12.152001 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b70a1987-5b32-4c96-9a76-ddf1a7dd8ae3 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:27:12.154218 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 188/362] 10.222.0.57 () {58 vars in 1023 bytes} [Fri Jul 1 16:27:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 01 16:27:12.155139 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e85ee37-09f4-4759-9fd0-6144f81ddc58 None None] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:27:12.162645 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 175/363] 10.222.0.57 () {56 vars in 1111 bytes} [Fri Jul 1 16:27:11 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:27:12.166647 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70a1987-5b32-4c96-9a76-ddf1a7dd8ae3 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:27:12.271001 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:27:12.300327 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e85ee37-09f4-4759-9fd0-6144f81ddc58 None None] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:27:12.313423 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70a1987-5b32-4c96-9a76-ddf1a7dd8ae3 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:27:12.342884 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f9d2ca-a1e3-4550-92ea-736e55ec97ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:27:12.343844 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f9d2ca-a1e3-4550-92ea-736e55ec97ba None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:12.345003 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f9d2ca-a1e3-4550-92ea-736e55ec97ba None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:12.345125 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f9d2ca-a1e3-4550-92ea-736e55ec97ba None admin] PATH_INFO: `/v3/groups/eb554eeac000489bb0d030619756ec4b` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:12.346316 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f9d2ca-a1e3-4550-92ea-736e55ec97ba None admin] RBAC: Authorizing `identity:get_group(group_id=eb554eeac000489bb0d030619756ec4b)` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:27:12.347843 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f9d2ca-a1e3-4550-92ea-736e55ec97ba None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:27:12.349494 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 189/364] 10.222.0.57 () {56 vars in 1180 bytes} [Fri Jul 1 16:27:12 2022] GET /identity/v3/groups/eb554eeac000489bb0d030619756ec4b => generated 226 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:27:12.350260 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f9d2ca-a1e3-4550-92ea-736e55ec97ba None admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:27:12.363453 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46008189-f987-4bf3-a1fa-6366f096d32c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:27:12.364448 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46008189-f987-4bf3-a1fa-6366f096d32c None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:12.364895 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46008189-f987-4bf3-a1fa-6366f096d32c None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:12.365186 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46008189-f987-4bf3-a1fa-6366f096d32c None admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:12.365974 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46008189-f987-4bf3-a1fa-6366f096d32c None admin] RBAC: Adding query filter params (name=reader) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:27:12.366286 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46008189-f987-4bf3-a1fa-6366f096d32c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:27:12.367701 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46008189-f987-4bf3-a1fa-6366f096d32c None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:27:12.377334 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 176/365] 10.222.0.57 () {56 vars in 1101 bytes} [Fri Jul 1 16:27:12 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:27:12.377507 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46008189-f987-4bf3-a1fa-6366f096d32c None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:27:12.397054 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=100 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:27:12.480604 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f9d2ca-a1e3-4550-92ea-736e55ec97ba None admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:27:12.512712 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46008189-f987-4bf3-a1fa-6366f096d32c None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:27:12.525163 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c022ba2-901a-4c8d-a1b1-7df9ca2719bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:27:12.526447 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c022ba2-901a-4c8d-a1b1-7df9ca2719bd None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:12.526879 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c022ba2-901a-4c8d-a1b1-7df9ca2719bd None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:12.527304 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c022ba2-901a-4c8d-a1b1-7df9ca2719bd None admin] PATH_INFO: `/v3/groups/eb554eeac000489bb0d030619756ec4b` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:12.530098 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c022ba2-901a-4c8d-a1b1-7df9ca2719bd None admin] RBAC: Authorizing `identity:get_group(group_id=eb554eeac000489bb0d030619756ec4b)` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:27:12.531770 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c022ba2-901a-4c8d-a1b1-7df9ca2719bd None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:27:12.533709 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 190/366] 10.222.0.57 () {56 vars in 1180 bytes} [Fri Jul 1 16:27:12 2022] GET /identity/v3/groups/eb554eeac000489bb0d030619756ec4b => generated 226 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:27:12.534686 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c022ba2-901a-4c8d-a1b1-7df9ca2719bd None admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:27:12.559475 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b49f68-eaee-4bb9-b945-42ce6a46fc70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:27:12.560136 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b49f68-eaee-4bb9-b945-42ce6a46fc70 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:12.560422 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b49f68-eaee-4bb9-b945-42ce6a46fc70 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:12.560704 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b49f68-eaee-4bb9-b945-42ce6a46fc70 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:12.569315 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:27:12.648115 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b49f68-eaee-4bb9-b945-42ce6a46fc70 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:27:12.650038 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b49f68-eaee-4bb9-b945-42ce6a46fc70 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:27:12.668690 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c022ba2-901a-4c8d-a1b1-7df9ca2719bd None admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:27:12.672340 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=100 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:27:12.716295 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02fc3ced-590d-4d14-873a-0a1f247b8415 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:27:12.717120 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fc3ced-590d-4d14-873a-0a1f247b8415 None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:12.717410 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fc3ced-590d-4d14-873a-0a1f247b8415 None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:12.717694 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fc3ced-590d-4d14-873a-0a1f247b8415 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:12.732671 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02fc3ced-590d-4d14-873a-0a1f247b8415 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 01 16:27:12.739802 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application [None req-02fc3ced-590d-4d14-873a-0a1f247b8415 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 01 16:27:12.739802 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 16:27:12.739802 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 16:27:12.739802 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 16:27:12.739802 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 16:27:12.739802 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 16:27:12.739802 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 16:27:12.739802 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 16:27:12.739802 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 16:27:12.739802 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 16:27:12.739802 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 16:27:12.739802 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 16:27:12.739802 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 01 16:27:12.739802 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 01 16:27:12.739802 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 01 16:27:12.739802 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 01 16:27:12.740737 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 01 16:27:12.740737 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 01 16:27:12.740737 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 01 16:27:12.740737 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 01 16:27:12.740737 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 16:27:12.740737 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 16:27:12.740737 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 16:27:12.740737 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 16:27:12.740737 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 16:27:12.740737 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 16:27:12.740737 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 16:27:12.740737 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application with Lock( Jul 01 16:27:12.740737 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 16:27:12.740737 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self._enter() Jul 01 16:27:12.740737 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 16:27:12.740737 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 16:27:12.741530 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 16:27:12.741530 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.creator() Jul 01 16:27:12.741530 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 16:27:12.741530 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application created_value = creator( Jul 01 16:27:12.741530 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 01 16:27:12.741530 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 01 16:27:12.741530 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 01 16:27:12.741530 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 01 16:27:12.741530 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 01 16:27:12.741530 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 01 16:27:12.741530 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 01 16:27:12.741530 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application Jul 01 16:27:12.741530 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 191/367] 10.222.0.57 () {56 vars in 1096 bytes} [Fri Jul 1 16:27:12 2022] GET /identity/v3/roles/admin => generated 83 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 16:27:12.742190 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02fc3ced-590d-4d14-873a-0a1f247b8415 None admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:27:12.784721 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 177/368] 10.222.0.57 () {58 vars in 1282 bytes} [Fri Jul 1 16:27:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 408 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 01 16:27:12.786006 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b49f68-eaee-4bb9-b945-42ce6a46fc70 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:27:12.831654 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:27:12.890760 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02fc3ced-590d-4d14-873a-0a1f247b8415 None admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:27:12.928461 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b49f68-eaee-4bb9-b945-42ce6a46fc70 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:27:12.938833 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609591ef-8742-4c49-9673-bdf94a680989 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:27:12.939957 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609591ef-8742-4c49-9673-bdf94a680989 None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:12.940445 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609591ef-8742-4c49-9673-bdf94a680989 None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:12.941073 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609591ef-8742-4c49-9673-bdf94a680989 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:12.941994 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-609591ef-8742-4c49-9673-bdf94a680989 None admin] RBAC: Adding query filter params (group.id=eb554eeac000489bb0d030619756ec4b, role.id=983be40bdab04c478eaa90c6b6a46b14, scope.project.id=63f84f2aa0bf4707bd9551422f0cae3e) {{(pid=80312) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:27:12.942398 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-609591ef-8742-4c49-9673-bdf94a680989 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:27:12.944044 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-609591ef-8742-4c49-9673-bdf94a680989 None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:27:12.952955 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 192/369] 10.222.0.57 () {56 vars in 1376 bytes} [Fri Jul 1 16:27:12 2022] GET /identity/v3/role_assignments?group.id=eb554eeac000489bb0d030619756ec4b&scope.project.id=63f84f2aa0bf4707bd9551422f0cae3e&role.id=983be40bdab04c478eaa90c6b6a46b14 => generated 260 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:27:12.953700 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609591ef-8742-4c49-9673-bdf94a680989 None admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:27:12.972671 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778b5342-3b77-477a-bd02-161189268292 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:27:12.973647 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778b5342-3b77-477a-bd02-161189268292 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:12.973850 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778b5342-3b77-477a-bd02-161189268292 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:12.974130 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778b5342-3b77-477a-bd02-161189268292 None admin] PATH_INFO: `/v3/users/9f0dcfde5e924ebcb328cef131b4bb34` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:12.975387 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-778b5342-3b77-477a-bd02-161189268292 None admin] RBAC: Authorizing `identity:get_user(user_id=9f0dcfde5e924ebcb328cef131b4bb34)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:27:12.976878 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-778b5342-3b77-477a-bd02-161189268292 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:27:12.978755 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 178/370] 10.222.0.57 () {56 vars in 1177 bytes} [Fri Jul 1 16:27:12 2022] GET /identity/v3/users/9f0dcfde5e924ebcb328cef131b4bb34 => generated 293 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:27:12.979578 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778b5342-3b77-477a-bd02-161189268292 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:27:13.004311 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=100 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:27:13.088862 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609591ef-8742-4c49-9673-bdf94a680989 None admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:27:13.115751 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778b5342-3b77-477a-bd02-161189268292 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:27:13.117977 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:27:13.131841 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607658b5-a1c3-4e62-95ef-50d6e9ca5b43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:27:13.132628 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607658b5-a1c3-4e62-95ef-50d6e9ca5b43 None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:13.132851 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607658b5-a1c3-4e62-95ef-50d6e9ca5b43 None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:13.133259 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607658b5-a1c3-4e62-95ef-50d6e9ca5b43 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:13.134652 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-607658b5-a1c3-4e62-95ef-50d6e9ca5b43 None admin] RBAC: Adding query filter params (group.id=eb554eeac000489bb0d030619756ec4b, role.id=a8e5c73da81d4f0eb7b92c7058e500fe, scope.project.id=5bef204363a04d5d8ceab527b8803f6e) {{(pid=80312) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:27:13.134652 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-607658b5-a1c3-4e62-95ef-50d6e9ca5b43 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:27:13.136539 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-607658b5-a1c3-4e62-95ef-50d6e9ca5b43 None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:27:13.145335 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 193/371] 10.222.0.57 () {56 vars in 1376 bytes} [Fri Jul 1 16:27:12 2022] GET /identity/v3/role_assignments?group.id=eb554eeac000489bb0d030619756ec4b&scope.project.id=5bef204363a04d5d8ceab527b8803f6e&role.id=a8e5c73da81d4f0eb7b92c7058e500fe => generated 260 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:27:13.149134 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607658b5-a1c3-4e62-95ef-50d6e9ca5b43 None admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:27:13.161316 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5886e1f7-a131-4a3b-822a-2f2df1401b05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:27:13.162388 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5886e1f7-a131-4a3b-822a-2f2df1401b05 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:13.162884 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5886e1f7-a131-4a3b-822a-2f2df1401b05 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:13.163480 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5886e1f7-a131-4a3b-822a-2f2df1401b05 None admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:13.164492 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5886e1f7-a131-4a3b-822a-2f2df1401b05 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:27:13.165701 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5886e1f7-a131-4a3b-822a-2f2df1401b05 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:27:13.168189 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5886e1f7-a131-4a3b-822a-2f2df1401b05 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:27:13.177993 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 179/372] 10.222.0.57 () {56 vars in 1099 bytes} [Fri Jul 1 16:27:12 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:27:13.180337 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5886e1f7-a131-4a3b-822a-2f2df1401b05 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:27:13.273118 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=100 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:27:13.282205 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607658b5-a1c3-4e62-95ef-50d6e9ca5b43 None admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:27:13.323195 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d38337f-b2ff-450a-8823-ac1324c23419 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:27:13.324087 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d38337f-b2ff-450a-8823-ac1324c23419 None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:13.324374 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d38337f-b2ff-450a-8823-ac1324c23419 None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:13.324653 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d38337f-b2ff-450a-8823-ac1324c23419 None admin] PATH_INFO: `/v3/projects/63f84f2aa0bf4707bd9551422f0cae3e` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:13.326034 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d38337f-b2ff-450a-8823-ac1324c23419 None admin] RBAC: Authorizing `identity:get_project(project_id=63f84f2aa0bf4707bd9551422f0cae3e)` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:27:13.327636 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d38337f-b2ff-450a-8823-ac1324c23419 None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:27:13.331255 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 194/373] 10.222.0.57 () {56 vars in 1186 bytes} [Fri Jul 1 16:27:13 2022] GET /identity/v3/projects/63f84f2aa0bf4707bd9551422f0cae3e => generated 302 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:27:13.334485 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d38337f-b2ff-450a-8823-ac1324c23419 None admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:27:13.336700 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5886e1f7-a131-4a3b-822a-2f2df1401b05 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:27:13.397351 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619b9e11-b5c6-4d5b-a405-70a407a69339 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:27:13.398024 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619b9e11-b5c6-4d5b-a405-70a407a69339 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:13.398582 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619b9e11-b5c6-4d5b-a405-70a407a69339 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:13.398883 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619b9e11-b5c6-4d5b-a405-70a407a69339 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:13.399803 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619b9e11-b5c6-4d5b-a405-70a407a69339 None admin] RBAC: Adding query filter params (role.id=bd7cefce73d14bd7827b58b6bfbe5e7a, user.id=9f0dcfde5e924ebcb328cef131b4bb34, scope.system=all) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:27:13.400507 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619b9e11-b5c6-4d5b-a405-70a407a69339 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:27:13.401986 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619b9e11-b5c6-4d5b-a405-70a407a69339 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:27:13.422438 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 180/374] 10.222.0.57 () {56 vars in 1308 bytes} [Fri Jul 1 16:27:13 2022] GET /identity/v3/role_assignments?user.id=9f0dcfde5e924ebcb328cef131b4bb34&scope.system=all&role.id=bd7cefce73d14bd7827b58b6bfbe5e7a => generated 510 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:27:13.423930 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39dd047-172f-4fbf-89d3-c97dcb729dfd None None] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:13.424224 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39dd047-172f-4fbf-89d3-c97dcb729dfd None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:13.424503 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39dd047-172f-4fbf-89d3-c97dcb729dfd None None] PATH_INFO: `/` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:13.426220 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 181/375] 10.222.0.57 () {54 vars in 926 bytes} [Fri Jul 1 16:27:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 16:27:13.428930 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39dd047-172f-4fbf-89d3-c97dcb729dfd None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:27:13.484562 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d38337f-b2ff-450a-8823-ac1324c23419 None admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:27:13.488381 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:27:13.551455 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a7e5c1-58f9-47ed-beff-f3d1969032d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:27:13.552132 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a7e5c1-58f9-47ed-beff-f3d1969032d2 None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:13.553611 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a7e5c1-58f9-47ed-beff-f3d1969032d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:13.553906 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a7e5c1-58f9-47ed-beff-f3d1969032d2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:13.608930 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=100 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:27:13.636570 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39dd047-172f-4fbf-89d3-c97dcb729dfd None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:27:13.661195 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a7e5c1-58f9-47ed-beff-f3d1969032d2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:27:13.662900 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a7e5c1-58f9-47ed-beff-f3d1969032d2 None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:27:13.681189 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f02279-c959-4e35-9e6f-43a37d9d4f43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:27:13.682562 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f02279-c959-4e35-9e6f-43a37d9d4f43 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:13.682991 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f02279-c959-4e35-9e6f-43a37d9d4f43 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:13.683278 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f02279-c959-4e35-9e6f-43a37d9d4f43 None admin] PATH_INFO: `/v3/groups/eb554eeac000489bb0d030619756ec4b` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:13.684667 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f02279-c959-4e35-9e6f-43a37d9d4f43 None admin] RBAC: Authorizing `identity:get_group(group_id=eb554eeac000489bb0d030619756ec4b)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:27:13.686856 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f02279-c959-4e35-9e6f-43a37d9d4f43 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:27:13.688839 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 182/376] 10.222.0.57 () {56 vars in 1180 bytes} [Fri Jul 1 16:27:13 2022] GET /identity/v3/groups/eb554eeac000489bb0d030619756ec4b => generated 226 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:27:13.690741 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44462f1-632e-4586-87b0-f492b8eb37f4 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:13.691191 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44462f1-632e-4586-87b0-f492b8eb37f4 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:13.691591 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44462f1-632e-4586-87b0-f492b8eb37f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:13.715905 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-b44462f1-632e-4586-87b0-f492b8eb37f4 None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:27:13.717508 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-b44462f1-632e-4586-87b0-f492b8eb37f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:27:13.790118 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 195/377] 10.222.0.57 () {58 vars in 1282 bytes} [Fri Jul 1 16:27:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 458 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 01 16:27:13.791091 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a7e5c1-58f9-47ed-beff-f3d1969032d2 None admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:27:13.796514 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 183/378] 10.222.0.57 () {58 vars in 1023 bytes} [Fri Jul 1 16:27:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 01 16:27:13.800597 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44462f1-632e-4586-87b0-f492b8eb37f4 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:27:13.828835 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:27:13.886756 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=100 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:27:13.943827 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a7e5c1-58f9-47ed-beff-f3d1969032d2 None admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:27:13.945350 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44462f1-632e-4586-87b0-f492b8eb37f4 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:27:13.989457 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb1e5a9-bb68-4532-9181-6de076da37a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:27:13.990521 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb1e5a9-bb68-4532-9181-6de076da37a8 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:13.990945 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb1e5a9-bb68-4532-9181-6de076da37a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:13.991231 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59878e7b-7050-4b92-b137-4743bfe76600 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:27:13.991392 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb1e5a9-bb68-4532-9181-6de076da37a8 None admin] PATH_INFO: `/v3/projects/62a34c02685249b399e8facc39cc11f8` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:13.991902 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59878e7b-7050-4b92-b137-4743bfe76600 None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:13.992301 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59878e7b-7050-4b92-b137-4743bfe76600 None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:13.992594 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59878e7b-7050-4b92-b137-4743bfe76600 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:13.992911 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb1e5a9-bb68-4532-9181-6de076da37a8 None admin] RBAC: Authorizing `identity:get_project(project_id=62a34c02685249b399e8facc39cc11f8)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:27:13.993696 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59878e7b-7050-4b92-b137-4743bfe76600 None admin] RBAC: Adding query filter params (group.id=eb554eeac000489bb0d030619756ec4b, role.id=a8e5c73da81d4f0eb7b92c7058e500fe, scope.project.id=63f84f2aa0bf4707bd9551422f0cae3e) {{(pid=80312) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:27:13.994254 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59878e7b-7050-4b92-b137-4743bfe76600 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:27:13.994754 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb1e5a9-bb68-4532-9181-6de076da37a8 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:27:13.996270 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59878e7b-7050-4b92-b137-4743bfe76600 None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:27:13.996717 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 184/379] 10.222.0.57 () {56 vars in 1186 bytes} [Fri Jul 1 16:27:13 2022] GET /identity/v3/projects/62a34c02685249b399e8facc39cc11f8 => generated 344 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:27:13.997392 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb1e5a9-bb68-4532-9181-6de076da37a8 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:27:14.003993 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 196/380] 10.222.0.57 () {56 vars in 1376 bytes} [Fri Jul 1 16:27:13 2022] GET /identity/v3/role_assignments?group.id=eb554eeac000489bb0d030619756ec4b&scope.project.id=63f84f2aa0bf4707bd9551422f0cae3e&role.id=a8e5c73da81d4f0eb7b92c7058e500fe => generated 260 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:27:14.004695 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59878e7b-7050-4b92-b137-4743bfe76600 None admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:27:14.176883 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:27:14.203500 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb1e5a9-bb68-4532-9181-6de076da37a8 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:27:14.214496 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59878e7b-7050-4b92-b137-4743bfe76600 None admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:27:14.249898 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=100 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:27:14.265597 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7b2951-96f6-431f-aba2-04119d622a15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:27:14.266794 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7b2951-96f6-431f-aba2-04119d622a15 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:14.267600 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7b2951-96f6-431f-aba2-04119d622a15 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:14.268087 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7b2951-96f6-431f-aba2-04119d622a15 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:14.281570 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce7b2951-96f6-431f-aba2-04119d622a15 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 01 16:27:14.291854 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51364cc2-10d7-4e10-b89f-161aa956659a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:27:14.292088 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application [None req-ce7b2951-96f6-431f-aba2-04119d622a15 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 01 16:27:14.292088 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 16:27:14.292088 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 16:27:14.292088 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 16:27:14.292088 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 16:27:14.292088 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 16:27:14.292088 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 16:27:14.292088 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 16:27:14.292088 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 16:27:14.292088 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 16:27:14.292088 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 16:27:14.292088 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 16:27:14.292088 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 01 16:27:14.292088 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 01 16:27:14.292088 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 01 16:27:14.292088 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 01 16:27:14.292922 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 01 16:27:14.292922 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 01 16:27:14.292922 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 01 16:27:14.292922 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 01 16:27:14.292922 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 16:27:14.292922 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 16:27:14.292922 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 16:27:14.292922 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 16:27:14.292922 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 16:27:14.292922 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 16:27:14.292922 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 16:27:14.292922 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application with Lock( Jul 01 16:27:14.292922 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 16:27:14.292922 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._enter() Jul 01 16:27:14.292922 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 16:27:14.292922 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 16:27:14.293733 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 16:27:14.293733 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.creator() Jul 01 16:27:14.293733 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 16:27:14.293733 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application created_value = creator( Jul 01 16:27:14.293733 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 01 16:27:14.293733 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 01 16:27:14.293733 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 01 16:27:14.293733 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 01 16:27:14.293733 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 01 16:27:14.293733 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 01 16:27:14.293733 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 01 16:27:14.293733 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Jul 01 16:27:14.294338 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51364cc2-10d7-4e10-b89f-161aa956659a None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:14.294338 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51364cc2-10d7-4e10-b89f-161aa956659a None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:14.294338 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51364cc2-10d7-4e10-b89f-161aa956659a None admin] PATH_INFO: `/v3/groups/1b0582f065d1420ea828d3d4082fdc87` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:14.297002 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 185/381] 10.222.0.57 () {56 vars in 1099 bytes} [Fri Jul 1 16:27:13 2022] GET /identity/v3/roles/member => generated 84 bytes in 300 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 16:27:14.303025 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7b2951-96f6-431f-aba2-04119d622a15 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:27:14.310105 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51364cc2-10d7-4e10-b89f-161aa956659a None admin] RBAC: Authorizing `identity:get_group(group_id=1b0582f065d1420ea828d3d4082fdc87)` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:27:14.311833 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51364cc2-10d7-4e10-b89f-161aa956659a None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:27:14.320684 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 197/382] 10.222.0.57 () {56 vars in 1180 bytes} [Fri Jul 1 16:27:14 2022] GET /identity/v3/groups/1b0582f065d1420ea828d3d4082fdc87 => generated 229 bytes in 309 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:27:14.322128 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51364cc2-10d7-4e10-b89f-161aa956659a None admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:27:14.437866 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7b2951-96f6-431f-aba2-04119d622a15 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:27:14.508198 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e98c748-f547-4136-879e-37c43801e0d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:27:14.509448 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e98c748-f547-4136-879e-37c43801e0d8 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:14.510088 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e98c748-f547-4136-879e-37c43801e0d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:14.510683 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e98c748-f547-4136-879e-37c43801e0d8 None admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:14.512221 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e98c748-f547-4136-879e-37c43801e0d8 None admin] RBAC: Adding query filter params (name=member) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:27:14.512895 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e98c748-f547-4136-879e-37c43801e0d8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:27:14.516811 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e98c748-f547-4136-879e-37c43801e0d8 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:27:14.527898 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 186/383] 10.222.0.57 () {56 vars in 1101 bytes} [Fri Jul 1 16:27:14 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 01 16:27:14.533489 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e98c748-f547-4136-879e-37c43801e0d8 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:27:14.552069 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:27:14.561196 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51364cc2-10d7-4e10-b89f-161aa956659a None admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:27:14.645685 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccee8d72-b4fc-49ba-9d95-4bb4f7e74301 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:27:14.647548 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccee8d72-b4fc-49ba-9d95-4bb4f7e74301 None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:14.649092 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccee8d72-b4fc-49ba-9d95-4bb4f7e74301 None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:14.650874 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccee8d72-b4fc-49ba-9d95-4bb4f7e74301 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:14.652240 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccee8d72-b4fc-49ba-9d95-4bb4f7e74301 None admin] RBAC: Adding query filter params (group.id=1b0582f065d1420ea828d3d4082fdc87, role.id=84ee739146934c1ab32c501ff8093644, scope.project.id=62a34c02685249b399e8facc39cc11f8) {{(pid=80312) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:27:14.654073 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccee8d72-b4fc-49ba-9d95-4bb4f7e74301 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:27:14.655837 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccee8d72-b4fc-49ba-9d95-4bb4f7e74301 None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:27:14.669115 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 198/384] 10.222.0.57 () {56 vars in 1376 bytes} [Fri Jul 1 16:27:14 2022] GET /identity/v3/role_assignments?group.id=1b0582f065d1420ea828d3d4082fdc87&scope.project.id=62a34c02685249b399e8facc39cc11f8&role.id=84ee739146934c1ab32c501ff8093644 => generated 260 bytes in 348 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 01 16:27:14.690499 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e98c748-f547-4136-879e-37c43801e0d8 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:27:14.747273 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76293ff-e735-492f-a058-a680d4217a4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:27:14.748550 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76293ff-e735-492f-a058-a680d4217a4c None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:14.749036 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76293ff-e735-492f-a058-a680d4217a4c None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:14.749509 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76293ff-e735-492f-a058-a680d4217a4c None admin] PATH_INFO: `/v3/groups/eb554eeac000489bb0d030619756ec4b` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:14.751104 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a76293ff-e735-492f-a058-a680d4217a4c None admin] RBAC: Authorizing `identity:get_group(group_id=eb554eeac000489bb0d030619756ec4b)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:27:14.752845 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a76293ff-e735-492f-a058-a680d4217a4c None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:27:14.755151 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 187/385] 10.222.0.57 () {56 vars in 1180 bytes} [Fri Jul 1 16:27:14 2022] GET /identity/v3/groups/eb554eeac000489bb0d030619756ec4b => generated 226 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 01 16:27:14.764421 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccee8d72-b4fc-49ba-9d95-4bb4f7e74301 None admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:27:14.848342 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=100 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:27:14.912574 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccee8d72-b4fc-49ba-9d95-4bb4f7e74301 None admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:27:14.949187 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a087b007-c698-408d-be54-f22230f08863 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:27:14.949984 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a087b007-c698-408d-be54-f22230f08863 None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:14.950270 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a087b007-c698-408d-be54-f22230f08863 None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:14.950581 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a087b007-c698-408d-be54-f22230f08863 None admin] PATH_INFO: `/v3/projects/5bef204363a04d5d8ceab527b8803f6e` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:14.951770 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a087b007-c698-408d-be54-f22230f08863 None admin] RBAC: Authorizing `identity:get_project(project_id=5bef204363a04d5d8ceab527b8803f6e)` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:27:14.953231 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a087b007-c698-408d-be54-f22230f08863 None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:27:14.955438 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 199/386] 10.222.0.57 () {56 vars in 1186 bytes} [Fri Jul 1 16:27:14 2022] GET /identity/v3/projects/5bef204363a04d5d8ceab527b8803f6e => generated 298 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:27:14.960929 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76293ff-e735-492f-a058-a680d4217a4c None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:27:15.057068 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:27:15.087362 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76293ff-e735-492f-a058-a680d4217a4c None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:27:15.141771 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52457f6c-8402-4c89-ad00-6150a94c06ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:27:15.142710 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52457f6c-8402-4c89-ad00-6150a94c06ce None admin] REQUEST_METHOD: `PUT` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:15.142997 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52457f6c-8402-4c89-ad00-6150a94c06ce None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:15.143272 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52457f6c-8402-4c89-ad00-6150a94c06ce None admin] PATH_INFO: `/v3/projects/5bef204363a04d5d8ceab527b8803f6e/groups/eb554eeac000489bb0d030619756ec4b/roles/983be40bdab04c478eaa90c6b6a46b14` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:15.145287 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52457f6c-8402-4c89-ad00-6150a94c06ce None admin] RBAC: Authorizing `identity:create_grant(project_id=5bef204363a04d5d8ceab527b8803f6e, group_id=eb554eeac000489bb0d030619756ec4b, role_id=983be40bdab04c478eaa90c6b6a46b14)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:27:15.146756 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52457f6c-8402-4c89-ad00-6150a94c06ce None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:27:15.161378 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 188/387] 10.222.0.57 () {58 vars in 1442 bytes} [Fri Jul 1 16:27:14 2022] PUT /identity/v3/projects/5bef204363a04d5d8ceab527b8803f6e/groups/eb554eeac000489bb0d030619756ec4b/roles/983be40bdab04c478eaa90c6b6a46b14 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 16:27:15.480723 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f59e8f6-987c-4599-a266-63d22d85fc3b None None] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:15.481188 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f59e8f6-987c-4599-a266-63d22d85fc3b None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:15.481317 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f59e8f6-987c-4599-a266-63d22d85fc3b None None] PATH_INFO: `/` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:15.483963 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 200/388] 10.222.0.57 () {54 vars in 926 bytes} [Fri Jul 1 16:27:15 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 01 16:27:15.489748 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d7c63d-811a-4507-b291-941f80ef0a49 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:15.490227 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d7c63d-811a-4507-b291-941f80ef0a49 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:15.490304 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d7c63d-811a-4507-b291-941f80ef0a49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:15.513472 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-47d7c63d-811a-4507-b291-941f80ef0a49 None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:27:15.514706 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-47d7c63d-811a-4507-b291-941f80ef0a49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:27:15.601568 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 189/389] 10.222.0.57 () {58 vars in 1023 bytes} [Fri Jul 1 16:27:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 01 16:27:15.644918 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f59e8f6-987c-4599-a266-63d22d85fc3b None None] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:27:15.783997 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f59e8f6-987c-4599-a266-63d22d85fc3b None None] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:27:15.808612 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=100 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:27:15.830006 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc45de6d-4e1c-428f-8d1f-9c108002711f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:27:15.830839 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc45de6d-4e1c-428f-8d1f-9c108002711f None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:15.831121 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc45de6d-4e1c-428f-8d1f-9c108002711f None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:15.831389 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc45de6d-4e1c-428f-8d1f-9c108002711f None admin] PATH_INFO: `/v3/roles/member` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:15.846827 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc45de6d-4e1c-428f-8d1f-9c108002711f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 01 16:27:15.855520 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application [None req-fc45de6d-4e1c-428f-8d1f-9c108002711f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 01 16:27:15.855520 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 16:27:15.855520 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 16:27:15.855520 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 16:27:15.855520 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 16:27:15.855520 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 16:27:15.855520 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 16:27:15.855520 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 16:27:15.855520 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 16:27:15.855520 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 16:27:15.855520 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 16:27:15.855520 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 16:27:15.855520 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 01 16:27:15.855520 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 01 16:27:15.855520 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 01 16:27:15.855520 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 01 16:27:15.856453 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 01 16:27:15.856453 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 01 16:27:15.856453 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 01 16:27:15.856453 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 01 16:27:15.856453 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 16:27:15.856453 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 16:27:15.856453 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 16:27:15.856453 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 16:27:15.856453 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 16:27:15.856453 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 16:27:15.856453 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 16:27:15.856453 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application with Lock( Jul 01 16:27:15.856453 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 16:27:15.856453 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self._enter() Jul 01 16:27:15.856453 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 16:27:15.856453 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 16:27:15.857309 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 16:27:15.857309 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.creator() Jul 01 16:27:15.857309 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 16:27:15.857309 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application created_value = creator( Jul 01 16:27:15.857309 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 01 16:27:15.857309 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 01 16:27:15.857309 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 01 16:27:15.857309 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 01 16:27:15.857309 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 01 16:27:15.857309 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 01 16:27:15.857309 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 01 16:27:15.857309 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application Jul 01 16:27:15.857309 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 201/390] 10.222.0.57 () {56 vars in 1099 bytes} [Fri Jul 1 16:27:15 2022] GET /identity/v3/roles/member => generated 84 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 16:27:15.864485 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d7c63d-811a-4507-b291-941f80ef0a49 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:27:15.877045 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529f2498-5ea8-470b-bda3-6fe26276f062 None None] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:15.877330 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529f2498-5ea8-470b-bda3-6fe26276f062 None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:15.877600 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529f2498-5ea8-470b-bda3-6fe26276f062 None None] PATH_INFO: `/` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:15.879276 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 202/391] 10.222.0.57 () {54 vars in 926 bytes} [Fri Jul 1 16:27:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 16:27:15.884427 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fd6dc8-2269-4a46-8c5b-459996286967 None None] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:15.884749 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fd6dc8-2269-4a46-8c5b-459996286967 None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:15.884994 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fd6dc8-2269-4a46-8c5b-459996286967 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:15.913467 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.auth.core [None req-e9fd6dc8-2269-4a46-8c5b-459996286967 None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80312) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:27:15.914699 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-e9fd6dc8-2269-4a46-8c5b-459996286967 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:27:15.981412 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:27:16.020603 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d7c63d-811a-4507-b291-941f80ef0a49 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:27:16.053011 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 203/392] 10.222.0.57 () {58 vars in 1023 bytes} [Fri Jul 1 16:27:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 01 16:27:16.065756 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ba2bca-e99b-4f03-805f-ff65c88fd29b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:27:16.066475 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ba2bca-e99b-4f03-805f-ff65c88fd29b None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:16.066766 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ba2bca-e99b-4f03-805f-ff65c88fd29b None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:16.067052 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ba2bca-e99b-4f03-805f-ff65c88fd29b None admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:16.067820 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ba2bca-e99b-4f03-805f-ff65c88fd29b None admin] RBAC: Adding query filter params (name=member) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:27:16.068132 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ba2bca-e99b-4f03-805f-ff65c88fd29b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:27:16.069552 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ba2bca-e99b-4f03-805f-ff65c88fd29b None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:27:16.080611 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9fd6dc8-2269-4a46-8c5b-459996286967 None None] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:27:16.085981 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 190/393] 10.222.0.57 () {56 vars in 1101 bytes} [Fri Jul 1 16:27:15 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:27:16.097048 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ba2bca-e99b-4f03-805f-ff65c88fd29b None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:27:16.241457 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9fd6dc8-2269-4a46-8c5b-459996286967 None None] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:27:16.259385 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=100 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:27:16.284525 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0dbbb93-461b-4b7c-a478-708c5de19054 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:27:16.286703 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0dbbb93-461b-4b7c-a478-708c5de19054 None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:16.286999 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0dbbb93-461b-4b7c-a478-708c5de19054 None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:16.287286 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0dbbb93-461b-4b7c-a478-708c5de19054 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:16.300274 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ba2bca-e99b-4f03-805f-ff65c88fd29b None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:27:16.315233 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0dbbb93-461b-4b7c-a478-708c5de19054 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 01 16:27:16.326323 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application [None req-e0dbbb93-461b-4b7c-a478-708c5de19054 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 01 16:27:16.326323 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 16:27:16.326323 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 16:27:16.326323 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 16:27:16.326323 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 16:27:16.326323 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 16:27:16.326323 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 16:27:16.326323 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 16:27:16.326323 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 16:27:16.326323 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 16:27:16.326323 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 16:27:16.326323 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 16:27:16.326323 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 01 16:27:16.326323 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 01 16:27:16.326323 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 01 16:27:16.326323 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 01 16:27:16.327511 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 01 16:27:16.327511 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 01 16:27:16.327511 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 01 16:27:16.327511 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 01 16:27:16.327511 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 16:27:16.327511 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 16:27:16.327511 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 16:27:16.327511 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 16:27:16.327511 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 16:27:16.327511 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 16:27:16.327511 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 16:27:16.327511 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application with Lock( Jul 01 16:27:16.327511 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 16:27:16.327511 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self._enter() Jul 01 16:27:16.327511 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 16:27:16.327511 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 16:27:16.328444 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 16:27:16.328444 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.creator() Jul 01 16:27:16.328444 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 16:27:16.328444 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application created_value = creator( Jul 01 16:27:16.328444 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 01 16:27:16.328444 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 01 16:27:16.328444 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 01 16:27:16.328444 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 01 16:27:16.328444 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 01 16:27:16.328444 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 01 16:27:16.328444 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 01 16:27:16.328444 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application Jul 01 16:27:16.328444 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 204/394] 10.222.0.57 () {56 vars in 1114 bytes} [Fri Jul 1 16:27:16 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 01 16:27:16.335895 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0dbbb93-461b-4b7c-a478-708c5de19054 None admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:27:16.375085 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:27:16.394593 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3315e0f-f347-4c04-8cfd-fe735f2213ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:27:16.395565 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3315e0f-f347-4c04-8cfd-fe735f2213ac None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:16.395920 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3315e0f-f347-4c04-8cfd-fe735f2213ac None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:16.396203 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3315e0f-f347-4c04-8cfd-fe735f2213ac None admin] PATH_INFO: `/v3/groups/eb554eeac000489bb0d030619756ec4b` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:16.397403 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3315e0f-f347-4c04-8cfd-fe735f2213ac None admin] RBAC: Authorizing `identity:get_group(group_id=eb554eeac000489bb0d030619756ec4b)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:27:16.398894 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3315e0f-f347-4c04-8cfd-fe735f2213ac None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:27:16.410847 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 191/395] 10.222.0.57 () {56 vars in 1180 bytes} [Fri Jul 1 16:27:16 2022] GET /identity/v3/groups/eb554eeac000489bb0d030619756ec4b => generated 226 bytes in 316 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 01 16:27:16.424014 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3315e0f-f347-4c04-8cfd-fe735f2213ac None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:27:16.456723 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0dbbb93-461b-4b7c-a478-708c5de19054 None admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:27:16.496176 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099a9bb8-9635-4546-8e9b-aae9f40c0813 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:27:16.496859 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099a9bb8-9635-4546-8e9b-aae9f40c0813 None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:16.497145 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099a9bb8-9635-4546-8e9b-aae9f40c0813 None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:16.497424 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099a9bb8-9635-4546-8e9b-aae9f40c0813 None admin] PATH_INFO: `/v3/roles` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:16.499172 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-099a9bb8-9635-4546-8e9b-aae9f40c0813 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80312) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:27:16.499172 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-099a9bb8-9635-4546-8e9b-aae9f40c0813 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:27:16.500064 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-099a9bb8-9635-4546-8e9b-aae9f40c0813 None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:27:16.510617 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 205/396] 10.222.0.57 () {56 vars in 1111 bytes} [Fri Jul 1 16:27:16 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:27:16.518818 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099a9bb8-9635-4546-8e9b-aae9f40c0813 None admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:27:16.593078 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=100 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:27:16.608645 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3315e0f-f347-4c04-8cfd-fe735f2213ac None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:27:16.657406 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2516360-dc2f-411c-91b4-ae4d887f7a39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:27:16.658483 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2516360-dc2f-411c-91b4-ae4d887f7a39 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:16.658826 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2516360-dc2f-411c-91b4-ae4d887f7a39 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:16.659111 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2516360-dc2f-411c-91b4-ae4d887f7a39 None admin] PATH_INFO: `/v3/projects/63f84f2aa0bf4707bd9551422f0cae3e` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:16.661089 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2516360-dc2f-411c-91b4-ae4d887f7a39 None admin] RBAC: Authorizing `identity:get_project(project_id=63f84f2aa0bf4707bd9551422f0cae3e)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:27:16.662774 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2516360-dc2f-411c-91b4-ae4d887f7a39 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:27:16.666142 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 192/397] 10.222.0.57 () {56 vars in 1186 bytes} [Fri Jul 1 16:27:16 2022] GET /identity/v3/projects/63f84f2aa0bf4707bd9551422f0cae3e => generated 302 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 01 16:27:16.667451 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329542b8-7395-4475-a285-619d13a95c67 None None] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:16.667743 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329542b8-7395-4475-a285-619d13a95c67 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:16.668028 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329542b8-7395-4475-a285-619d13a95c67 None None] PATH_INFO: `/` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:16.673822 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 193/398] 10.222.0.57 () {54 vars in 926 bytes} [Fri Jul 1 16:27:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 16:27:16.674983 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329542b8-7395-4475-a285-619d13a95c67 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:27:16.679945 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099a9bb8-9635-4546-8e9b-aae9f40c0813 None admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:27:16.718186 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7435dc3-9e4b-45bb-b5ab-a64ed6eb184c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:27:16.719182 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7435dc3-9e4b-45bb-b5ab-a64ed6eb184c None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:16.719467 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7435dc3-9e4b-45bb-b5ab-a64ed6eb184c None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:16.719747 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7435dc3-9e4b-45bb-b5ab-a64ed6eb184c None admin] PATH_INFO: `/v3/groups/eb554eeac000489bb0d030619756ec4b` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:16.720930 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7435dc3-9e4b-45bb-b5ab-a64ed6eb184c None admin] RBAC: Authorizing `identity:get_group(group_id=eb554eeac000489bb0d030619756ec4b)` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:27:16.722419 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7435dc3-9e4b-45bb-b5ab-a64ed6eb184c None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:27:16.724419 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 206/399] 10.222.0.57 () {56 vars in 1180 bytes} [Fri Jul 1 16:27:16 2022] GET /identity/v3/groups/eb554eeac000489bb0d030619756ec4b => generated 226 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:27:16.725694 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026e6404-998b-4606-ad26-c50f071dbd24 None None] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:16.725986 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026e6404-998b-4606-ad26-c50f071dbd24 None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:16.726266 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026e6404-998b-4606-ad26-c50f071dbd24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:16.744412 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.auth.core [None req-026e6404-998b-4606-ad26-c50f071dbd24 None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80312) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:27:16.745541 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-026e6404-998b-4606-ad26-c50f071dbd24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:27:16.759438 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:27:16.808616 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329542b8-7395-4475-a285-619d13a95c67 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:27:16.826369 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 207/400] 10.222.0.57 () {58 vars in 1023 bytes} [Fri Jul 1 16:27:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 01 16:27:16.826369 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026e6404-998b-4606-ad26-c50f071dbd24 None None] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:27:16.880779 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=100 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:27:16.882540 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d09ed19-cc39-4325-9ed6-17f6619b3ee8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:27:16.884534 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d09ed19-cc39-4325-9ed6-17f6619b3ee8 None admin] REQUEST_METHOD: `PUT` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:16.884828 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d09ed19-cc39-4325-9ed6-17f6619b3ee8 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:16.885115 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d09ed19-cc39-4325-9ed6-17f6619b3ee8 None admin] PATH_INFO: `/v3/projects/63f84f2aa0bf4707bd9551422f0cae3e/groups/eb554eeac000489bb0d030619756ec4b/roles/983be40bdab04c478eaa90c6b6a46b14` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:16.887578 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d09ed19-cc39-4325-9ed6-17f6619b3ee8 None admin] RBAC: Authorizing `identity:create_grant(project_id=63f84f2aa0bf4707bd9551422f0cae3e, group_id=eb554eeac000489bb0d030619756ec4b, role_id=983be40bdab04c478eaa90c6b6a46b14)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:27:16.889261 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d09ed19-cc39-4325-9ed6-17f6619b3ee8 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:27:16.905266 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 194/401] 10.222.0.57 () {58 vars in 1442 bytes} [Fri Jul 1 16:27:16 2022] PUT /identity/v3/projects/63f84f2aa0bf4707bd9551422f0cae3e/groups/eb554eeac000489bb0d030619756ec4b/roles/983be40bdab04c478eaa90c6b6a46b14 => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 16:27:16.906487 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d09ed19-cc39-4325-9ed6-17f6619b3ee8 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:27:17.042718 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026e6404-998b-4606-ad26-c50f071dbd24 None None] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:27:17.096858 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d09ed19-cc39-4325-9ed6-17f6619b3ee8 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:27:17.106456 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91922974-2a38-4f0f-8b40-c431929f6d2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:27:17.107239 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91922974-2a38-4f0f-8b40-c431929f6d2b None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:17.107528 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91922974-2a38-4f0f-8b40-c431929f6d2b None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:17.107810 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91922974-2a38-4f0f-8b40-c431929f6d2b None admin] PATH_INFO: `/v3/projects/5bef204363a04d5d8ceab527b8803f6e` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:17.109025 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91922974-2a38-4f0f-8b40-c431929f6d2b None admin] RBAC: Authorizing `identity:get_project(project_id=5bef204363a04d5d8ceab527b8803f6e)` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:27:17.110500 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91922974-2a38-4f0f-8b40-c431929f6d2b None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:27:17.112510 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 208/402] 10.222.0.57 () {56 vars in 1186 bytes} [Fri Jul 1 16:27:16 2022] GET /identity/v3/projects/5bef204363a04d5d8ceab527b8803f6e => generated 298 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:27:17.118683 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91922974-2a38-4f0f-8b40-c431929f6d2b None admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:27:17.139592 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:27:17.143070 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d207c1-5f74-4113-9088-7903bd793731 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:27:17.146371 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d207c1-5f74-4113-9088-7903bd793731 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:17.146371 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d207c1-5f74-4113-9088-7903bd793731 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:17.146371 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d207c1-5f74-4113-9088-7903bd793731 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:17.162071 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d207c1-5f74-4113-9088-7903bd793731 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 01 16:27:17.172356 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application [None req-73d207c1-5f74-4113-9088-7903bd793731 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 01 16:27:17.172356 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 16:27:17.172356 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 16:27:17.172356 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 16:27:17.172356 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 16:27:17.172356 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 16:27:17.172356 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 16:27:17.172356 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 16:27:17.172356 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 16:27:17.172356 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 16:27:17.172356 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 16:27:17.172356 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 16:27:17.172356 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 01 16:27:17.172356 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 01 16:27:17.172356 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 01 16:27:17.172356 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 01 16:27:17.174014 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 01 16:27:17.174014 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 01 16:27:17.174014 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 01 16:27:17.174014 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 01 16:27:17.174014 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 16:27:17.174014 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 16:27:17.174014 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 16:27:17.174014 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 16:27:17.174014 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 16:27:17.174014 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 16:27:17.174014 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 16:27:17.174014 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application with Lock( Jul 01 16:27:17.174014 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 16:27:17.174014 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._enter() Jul 01 16:27:17.174014 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 16:27:17.174014 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 16:27:17.175388 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 16:27:17.175388 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.creator() Jul 01 16:27:17.175388 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 16:27:17.175388 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application created_value = creator( Jul 01 16:27:17.175388 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 01 16:27:17.175388 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 01 16:27:17.175388 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 01 16:27:17.175388 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 01 16:27:17.175388 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 01 16:27:17.175388 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 01 16:27:17.175388 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 01 16:27:17.175388 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Jul 01 16:27:17.175388 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 195/403] 10.222.0.57 () {56 vars in 1114 bytes} [Fri Jul 1 16:27:16 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 268 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 16:27:17.180265 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d207c1-5f74-4113-9088-7903bd793731 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:27:17.261058 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=100 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:27:17.276539 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91922974-2a38-4f0f-8b40-c431929f6d2b None admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:27:17.311340 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d207c1-5f74-4113-9088-7903bd793731 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:27:17.351709 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ff1d8b-603c-41d0-8d4d-04a7d0be696d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:27:17.351942 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5c7d0e-d9ea-4d72-939c-2db34c4b00e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:27:17.352803 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5c7d0e-d9ea-4d72-939c-2db34c4b00e4 None admin] REQUEST_METHOD: `PUT` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:17.353111 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5c7d0e-d9ea-4d72-939c-2db34c4b00e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:17.353470 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5c7d0e-d9ea-4d72-939c-2db34c4b00e4 None admin] PATH_INFO: `/v3/projects/5bef204363a04d5d8ceab527b8803f6e/groups/eb554eeac000489bb0d030619756ec4b/roles/a8e5c73da81d4f0eb7b92c7058e500fe` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:17.353818 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ff1d8b-603c-41d0-8d4d-04a7d0be696d None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:17.354412 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ff1d8b-603c-41d0-8d4d-04a7d0be696d None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:17.354964 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ff1d8b-603c-41d0-8d4d-04a7d0be696d None admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:17.355887 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb5c7d0e-d9ea-4d72-939c-2db34c4b00e4 None admin] RBAC: Authorizing `identity:create_grant(project_id=5bef204363a04d5d8ceab527b8803f6e, group_id=eb554eeac000489bb0d030619756ec4b, role_id=a8e5c73da81d4f0eb7b92c7058e500fe)` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:27:17.356210 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ff1d8b-603c-41d0-8d4d-04a7d0be696d None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:27:17.356782 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ff1d8b-603c-41d0-8d4d-04a7d0be696d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:27:17.357330 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb5c7d0e-d9ea-4d72-939c-2db34c4b00e4 None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:27:17.358584 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ff1d8b-603c-41d0-8d4d-04a7d0be696d None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:27:17.373151 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 196/404] 10.222.0.57 () {56 vars in 1111 bytes} [Fri Jul 1 16:27:17 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 01 16:27:17.374968 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4201ab87-db54-4ccd-8bc0-eb7059504cd6 None None] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:17.375460 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4201ab87-db54-4ccd-8bc0-eb7059504cd6 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:17.375996 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4201ab87-db54-4ccd-8bc0-eb7059504cd6 None None] PATH_INFO: `/` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:17.377456 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 197/405] 10.222.0.57 () {54 vars in 926 bytes} [Fri Jul 1 16:27:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 16:27:17.378615 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4201ab87-db54-4ccd-8bc0-eb7059504cd6 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:27:17.381710 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 209/406] 10.222.0.57 () {58 vars in 1442 bytes} [Fri Jul 1 16:27:17 2022] PUT /identity/v3/projects/5bef204363a04d5d8ceab527b8803f6e/groups/eb554eeac000489bb0d030619756ec4b/roles/a8e5c73da81d4f0eb7b92c7058e500fe => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 16:27:17.383297 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430842b3-f397-4023-8d86-e792543f3421 None None] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:17.384892 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430842b3-f397-4023-8d86-e792543f3421 None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:17.385445 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430842b3-f397-4023-8d86-e792543f3421 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:17.419055 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.auth.core [None req-430842b3-f397-4023-8d86-e792543f3421 None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80312) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:27:17.421210 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-430842b3-f397-4023-8d86-e792543f3421 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:27:17.490787 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:27:17.540483 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 210/407] 10.222.0.57 () {58 vars in 1023 bytes} [Fri Jul 1 16:27:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 01 16:27:17.564450 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430842b3-f397-4023-8d86-e792543f3421 None None] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:27:17.567605 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4201ab87-db54-4ccd-8bc0-eb7059504cd6 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:27:17.606372 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c61d536-65d8-4ccf-addb-77be25c048a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:27:17.607323 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c61d536-65d8-4ccf-addb-77be25c048a6 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:17.607540 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c61d536-65d8-4ccf-addb-77be25c048a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:17.607796 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c61d536-65d8-4ccf-addb-77be25c048a6 None admin] PATH_INFO: `/v3/groups/eb554eeac000489bb0d030619756ec4b` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:17.608965 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c61d536-65d8-4ccf-addb-77be25c048a6 None admin] RBAC: Authorizing `identity:get_group(group_id=eb554eeac000489bb0d030619756ec4b)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:27:17.610427 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c61d536-65d8-4ccf-addb-77be25c048a6 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:27:17.612387 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 198/408] 10.222.0.57 () {56 vars in 1180 bytes} [Fri Jul 1 16:27:17 2022] GET /identity/v3/groups/eb554eeac000489bb0d030619756ec4b => generated 226 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:27:17.617736 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c61d536-65d8-4ccf-addb-77be25c048a6 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:27:17.691206 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:27:17.704458 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430842b3-f397-4023-8d86-e792543f3421 None None] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:27:17.737332 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c61d536-65d8-4ccf-addb-77be25c048a6 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:27:17.745919 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd92364-a3c1-4ce5-be5f-2bb4f78c7d37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:27:17.746757 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd92364-a3c1-4ce5-be5f-2bb4f78c7d37 None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:17.747046 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd92364-a3c1-4ce5-be5f-2bb4f78c7d37 None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:17.747326 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd92364-a3c1-4ce5-be5f-2bb4f78c7d37 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:17.757769 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd92364-a3c1-4ce5-be5f-2bb4f78c7d37 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 01 16:27:17.759127 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:27:17.766614 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application [None req-2dd92364-a3c1-4ce5-be5f-2bb4f78c7d37 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 01 16:27:17.766614 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 16:27:17.766614 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 16:27:17.766614 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 16:27:17.766614 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 16:27:17.766614 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 16:27:17.766614 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 16:27:17.766614 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 16:27:17.766614 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 16:27:17.766614 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 16:27:17.766614 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 16:27:17.766614 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 16:27:17.766614 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 01 16:27:17.766614 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 01 16:27:17.766614 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 01 16:27:17.766614 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 01 16:27:17.767685 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 01 16:27:17.767685 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 01 16:27:17.767685 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 01 16:27:17.767685 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 01 16:27:17.767685 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 16:27:17.767685 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 16:27:17.767685 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 16:27:17.767685 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 16:27:17.767685 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 16:27:17.767685 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 16:27:17.767685 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 16:27:17.767685 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application with Lock( Jul 01 16:27:17.767685 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 16:27:17.767685 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self._enter() Jul 01 16:27:17.767685 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 16:27:17.767685 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 16:27:17.768687 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 16:27:17.768687 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.creator() Jul 01 16:27:17.768687 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 16:27:17.768687 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application created_value = creator( Jul 01 16:27:17.768687 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 01 16:27:17.768687 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 01 16:27:17.768687 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 01 16:27:17.768687 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 01 16:27:17.768687 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 01 16:27:17.768687 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 01 16:27:17.768687 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 01 16:27:17.768687 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application Jul 01 16:27:17.768687 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 211/409] 10.222.0.57 () {56 vars in 1096 bytes} [Fri Jul 1 16:27:17 2022] GET /identity/v3/roles/admin => generated 83 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 01 16:27:17.769497 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946b06f8-6d3a-440f-9b76-4517d6811553 None None] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:17.769790 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946b06f8-6d3a-440f-9b76-4517d6811553 None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:17.770068 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946b06f8-6d3a-440f-9b76-4517d6811553 None None] PATH_INFO: `/` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:17.775895 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 212/410] 10.222.0.57 () {54 vars in 926 bytes} [Fri Jul 1 16:27:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 16:27:17.775895 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946b06f8-6d3a-440f-9b76-4517d6811553 None None] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:27:17.794140 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a288cb-7fdc-4201-ad4c-25692bad8061 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:27:17.795531 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a288cb-7fdc-4201-ad4c-25692bad8061 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:17.795957 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a288cb-7fdc-4201-ad4c-25692bad8061 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:17.796482 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a288cb-7fdc-4201-ad4c-25692bad8061 None admin] PATH_INFO: `/v3/projects/63f84f2aa0bf4707bd9551422f0cae3e` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:17.798229 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a288cb-7fdc-4201-ad4c-25692bad8061 None admin] RBAC: Authorizing `identity:get_project(project_id=63f84f2aa0bf4707bd9551422f0cae3e)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:27:17.800261 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a288cb-7fdc-4201-ad4c-25692bad8061 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:27:17.803204 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 199/411] 10.222.0.57 () {56 vars in 1186 bytes} [Fri Jul 1 16:27:17 2022] GET /identity/v3/projects/63f84f2aa0bf4707bd9551422f0cae3e => generated 302 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 01 16:27:17.804881 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a110a7-61f0-49b6-9a77-925e80d2b130 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:17.805361 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a110a7-61f0-49b6-9a77-925e80d2b130 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:17.806301 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a110a7-61f0-49b6-9a77-925e80d2b130 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:17.827307 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-f7a110a7-61f0-49b6-9a77-925e80d2b130 None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:27:17.829236 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-f7a110a7-61f0-49b6-9a77-925e80d2b130 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:27:17.914090 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946b06f8-6d3a-440f-9b76-4517d6811553 None None] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:27:17.915183 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 200/412] 10.222.0.57 () {58 vars in 1023 bytes} [Fri Jul 1 16:27:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 01 16:27:17.916158 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a110a7-61f0-49b6-9a77-925e80d2b130 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:27:17.953426 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b85464-547c-41a9-9e36-8f8b55966826 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:27:17.954412 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b85464-547c-41a9-9e36-8f8b55966826 None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:17.955120 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b85464-547c-41a9-9e36-8f8b55966826 None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:17.955495 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b85464-547c-41a9-9e36-8f8b55966826 None admin] PATH_INFO: `/v3/roles` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:17.956354 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b85464-547c-41a9-9e36-8f8b55966826 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80312) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:27:17.956766 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b85464-547c-41a9-9e36-8f8b55966826 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:27:17.958491 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b85464-547c-41a9-9e36-8f8b55966826 None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:27:17.968250 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 213/413] 10.222.0.57 () {56 vars in 1099 bytes} [Fri Jul 1 16:27:17 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 01 16:27:17.969298 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b85464-547c-41a9-9e36-8f8b55966826 None admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:27:17.982758 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=100 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:27:18.035783 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a110a7-61f0-49b6-9a77-925e80d2b130 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:27:18.052591 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:27:18.081606 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9246c26a-b9ca-4e8e-be3e-5a58bcc4c990 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:27:18.083120 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9246c26a-b9ca-4e8e-be3e-5a58bcc4c990 None admin] REQUEST_METHOD: `PUT` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:18.083849 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9246c26a-b9ca-4e8e-be3e-5a58bcc4c990 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:18.084721 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9246c26a-b9ca-4e8e-be3e-5a58bcc4c990 None admin] PATH_INFO: `/v3/projects/63f84f2aa0bf4707bd9551422f0cae3e/groups/eb554eeac000489bb0d030619756ec4b/roles/a8e5c73da81d4f0eb7b92c7058e500fe` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:18.089412 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9246c26a-b9ca-4e8e-be3e-5a58bcc4c990 None admin] RBAC: Authorizing `identity:create_grant(project_id=63f84f2aa0bf4707bd9551422f0cae3e, group_id=eb554eeac000489bb0d030619756ec4b, role_id=a8e5c73da81d4f0eb7b92c7058e500fe)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:27:18.093184 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9246c26a-b9ca-4e8e-be3e-5a58bcc4c990 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:27:18.095386 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b85464-547c-41a9-9e36-8f8b55966826 None admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:27:18.112181 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 201/414] 10.222.0.57 () {58 vars in 1442 bytes} [Fri Jul 1 16:27:17 2022] PUT /identity/v3/projects/63f84f2aa0bf4707bd9551422f0cae3e/groups/eb554eeac000489bb0d030619756ec4b/roles/a8e5c73da81d4f0eb7b92c7058e500fe => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 16:27:18.113306 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9246c26a-b9ca-4e8e-be3e-5a58bcc4c990 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:27:18.174406 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e25a7d1-d81f-415c-bdce-dc9edf916bdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:27:18.175136 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e25a7d1-d81f-415c-bdce-dc9edf916bdf None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:18.175419 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e25a7d1-d81f-415c-bdce-dc9edf916bdf None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:18.175692 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e25a7d1-d81f-415c-bdce-dc9edf916bdf None admin] PATH_INFO: `/v3/roles/member` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:18.202252 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e25a7d1-d81f-415c-bdce-dc9edf916bdf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 01 16:27:18.219138 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application [None req-9e25a7d1-d81f-415c-bdce-dc9edf916bdf None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 01 16:27:18.219138 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 16:27:18.219138 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 16:27:18.219138 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 16:27:18.219138 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 16:27:18.219138 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 16:27:18.219138 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 16:27:18.219138 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 16:27:18.219138 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 16:27:18.219138 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 16:27:18.219138 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 16:27:18.219138 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 16:27:18.219138 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 01 16:27:18.219138 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 01 16:27:18.219138 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 01 16:27:18.219138 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 01 16:27:18.220172 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 01 16:27:18.220172 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 01 16:27:18.220172 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 01 16:27:18.220172 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 01 16:27:18.220172 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 16:27:18.220172 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 16:27:18.220172 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 16:27:18.220172 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 16:27:18.220172 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 16:27:18.220172 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 16:27:18.220172 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 16:27:18.220172 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application with Lock( Jul 01 16:27:18.220172 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 16:27:18.220172 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self._enter() Jul 01 16:27:18.220172 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 16:27:18.220172 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 16:27:18.220982 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 16:27:18.220982 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.creator() Jul 01 16:27:18.220982 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 16:27:18.220982 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application created_value = creator( Jul 01 16:27:18.220982 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 01 16:27:18.220982 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 01 16:27:18.220982 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 01 16:27:18.220982 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 01 16:27:18.220982 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 01 16:27:18.220982 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 01 16:27:18.220982 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 01 16:27:18.220982 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application Jul 01 16:27:18.220982 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 214/415] 10.222.0.57 () {56 vars in 1099 bytes} [Fri Jul 1 16:27:17 2022] GET /identity/v3/roles/member => generated 84 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 16:27:18.226683 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e25a7d1-d81f-415c-bdce-dc9edf916bdf None admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:27:18.251951 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9246c26a-b9ca-4e8e-be3e-5a58bcc4c990 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:27:18.298514 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25397767-8287-4660-b09f-f53dc8af7596 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:27:18.299901 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25397767-8287-4660-b09f-f53dc8af7596 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:18.300673 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25397767-8287-4660-b09f-f53dc8af7596 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:18.301151 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25397767-8287-4660-b09f-f53dc8af7596 None admin] PATH_INFO: `/v3/groups/1b0582f065d1420ea828d3d4082fdc87` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:18.302695 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25397767-8287-4660-b09f-f53dc8af7596 None admin] RBAC: Authorizing `identity:get_group(group_id=1b0582f065d1420ea828d3d4082fdc87)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:27:18.304469 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25397767-8287-4660-b09f-f53dc8af7596 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:27:18.307043 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 202/416] 10.222.0.57 () {56 vars in 1180 bytes} [Fri Jul 1 16:27:18 2022] GET /identity/v3/groups/1b0582f065d1420ea828d3d4082fdc87 => generated 229 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:27:18.312812 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25397767-8287-4660-b09f-f53dc8af7596 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:27:18.374721 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:27:18.404683 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=100 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:27:18.440634 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e25a7d1-d81f-415c-bdce-dc9edf916bdf None admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:27:18.465439 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25397767-8287-4660-b09f-f53dc8af7596 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:27:18.497752 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292de6aa-2322-4776-8722-9acfb42d7727 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:27:18.498607 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292de6aa-2322-4776-8722-9acfb42d7727 None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:18.498965 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292de6aa-2322-4776-8722-9acfb42d7727 None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:18.499317 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292de6aa-2322-4776-8722-9acfb42d7727 None admin] PATH_INFO: `/v3/roles` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:18.500136 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-292de6aa-2322-4776-8722-9acfb42d7727 None admin] RBAC: Adding query filter params (name=member) {{(pid=80312) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:27:18.500508 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-292de6aa-2322-4776-8722-9acfb42d7727 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:27:18.502028 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-292de6aa-2322-4776-8722-9acfb42d7727 None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:27:18.513135 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c959a53-e320-48e3-98bf-f7a717d822c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:27:18.514805 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c959a53-e320-48e3-98bf-f7a717d822c6 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:18.515590 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c959a53-e320-48e3-98bf-f7a717d822c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:18.516307 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c959a53-e320-48e3-98bf-f7a717d822c6 None admin] PATH_INFO: `/v3/projects/62a34c02685249b399e8facc39cc11f8` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:18.522058 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 215/417] 10.222.0.57 () {56 vars in 1101 bytes} [Fri Jul 1 16:27:18 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:27:18.522811 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c959a53-e320-48e3-98bf-f7a717d822c6 None admin] RBAC: Authorizing `identity:get_project(project_id=62a34c02685249b399e8facc39cc11f8)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:27:18.524569 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c959a53-e320-48e3-98bf-f7a717d822c6 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:27:18.526789 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 203/418] 10.222.0.57 () {56 vars in 1186 bytes} [Fri Jul 1 16:27:18 2022] GET /identity/v3/projects/62a34c02685249b399e8facc39cc11f8 => generated 344 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:27:18.528126 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292de6aa-2322-4776-8722-9acfb42d7727 None admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:27:18.532175 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c959a53-e320-48e3-98bf-f7a717d822c6 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:27:18.707071 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c959a53-e320-48e3-98bf-f7a717d822c6 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:27:18.721552 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:27:18.753322 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd3f93b-0e21-45a5-ae75-964373db1515 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:27:18.754539 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd3f93b-0e21-45a5-ae75-964373db1515 None admin] REQUEST_METHOD: `PUT` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:18.754998 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd3f93b-0e21-45a5-ae75-964373db1515 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:18.755472 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd3f93b-0e21-45a5-ae75-964373db1515 None admin] PATH_INFO: `/v3/projects/62a34c02685249b399e8facc39cc11f8/groups/1b0582f065d1420ea828d3d4082fdc87/roles/84ee739146934c1ab32c501ff8093644` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:18.757639 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd3f93b-0e21-45a5-ae75-964373db1515 None admin] RBAC: Authorizing `identity:create_grant(project_id=62a34c02685249b399e8facc39cc11f8, group_id=1b0582f065d1420ea828d3d4082fdc87, role_id=84ee739146934c1ab32c501ff8093644)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:27:18.760534 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd3f93b-0e21-45a5-ae75-964373db1515 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:27:18.770589 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292de6aa-2322-4776-8722-9acfb42d7727 None admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:27:18.792313 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 204/419] 10.222.0.57 () {58 vars in 1442 bytes} [Fri Jul 1 16:27:18 2022] PUT /identity/v3/projects/62a34c02685249b399e8facc39cc11f8/groups/1b0582f065d1420ea828d3d4082fdc87/roles/84ee739146934c1ab32c501ff8093644 => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 16:27:18.807773 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=100 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:27:18.814666 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3eed721-df9e-45f0-9f7a-eba08fea161e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:27:18.815734 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3eed721-df9e-45f0-9f7a-eba08fea161e None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:18.816171 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3eed721-df9e-45f0-9f7a-eba08fea161e None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:18.816616 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3eed721-df9e-45f0-9f7a-eba08fea161e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:18.907852 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3eed721-df9e-45f0-9f7a-eba08fea161e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:27:18.909378 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3eed721-df9e-45f0-9f7a-eba08fea161e None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:27:19.019402 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 216/420] 10.222.0.57 () {58 vars in 1282 bytes} [Fri Jul 1 16:27:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 491 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 01 16:27:19.025581 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd3f93b-0e21-45a5-ae75-964373db1515 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:27:19.135128 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd3f93b-0e21-45a5-ae75-964373db1515 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:27:19.173910 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb38d67b-087b-4f50-a29e-5e44348dc358 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:27:19.174844 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb38d67b-087b-4f50-a29e-5e44348dc358 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:19.175129 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb38d67b-087b-4f50-a29e-5e44348dc358 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:19.175405 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb38d67b-087b-4f50-a29e-5e44348dc358 None admin] PATH_INFO: `/v3/projects/5bef204363a04d5d8ceab527b8803f6e` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:19.177233 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb38d67b-087b-4f50-a29e-5e44348dc358 None admin] RBAC: Authorizing `identity:get_project(project_id=5bef204363a04d5d8ceab527b8803f6e)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:27:19.178704 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb38d67b-087b-4f50-a29e-5e44348dc358 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:27:19.180894 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 205/421] 10.222.0.57 () {56 vars in 1186 bytes} [Fri Jul 1 16:27:19 2022] GET /identity/v3/projects/5bef204363a04d5d8ceab527b8803f6e => generated 298 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:27:19.186257 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3eed721-df9e-45f0-9f7a-eba08fea161e None admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:27:19.221185 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=100 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:27:19.295533 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3eed721-df9e-45f0-9f7a-eba08fea161e None admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:27:19.338306 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35995da7-c40a-4ce1-a12d-a132971e9ac4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:27:19.339204 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35995da7-c40a-4ce1-a12d-a132971e9ac4 None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:19.339488 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35995da7-c40a-4ce1-a12d-a132971e9ac4 None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:19.339765 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35995da7-c40a-4ce1-a12d-a132971e9ac4 None admin] PATH_INFO: `/v3/groups/eb554eeac000489bb0d030619756ec4b` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:19.340953 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35995da7-c40a-4ce1-a12d-a132971e9ac4 None admin] RBAC: Authorizing `identity:get_group(group_id=eb554eeac000489bb0d030619756ec4b)` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:27:19.342425 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35995da7-c40a-4ce1-a12d-a132971e9ac4 None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:27:19.344060 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 217/422] 10.222.0.57 () {56 vars in 1180 bytes} [Fri Jul 1 16:27:19 2022] GET /identity/v3/groups/eb554eeac000489bb0d030619756ec4b => generated 226 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:27:19.350132 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb38d67b-087b-4f50-a29e-5e44348dc358 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:27:19.392499 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:27:19.470292 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb38d67b-087b-4f50-a29e-5e44348dc358 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:27:19.489228 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9511cc-ccc5-45ca-9e56-17c6dccfbcd3 None None] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:19.489554 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9511cc-ccc5-45ca-9e56-17c6dccfbcd3 None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:19.489788 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9511cc-ccc5-45ca-9e56-17c6dccfbcd3 None None] PATH_INFO: `/` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:19.491309 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 218/423] 10.222.0.57 () {54 vars in 926 bytes} [Fri Jul 1 16:27:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 16:27:19.498614 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014029a9-edc1-46b5-a078-e9ce2beea511 None None] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:19.498927 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014029a9-edc1-46b5-a078-e9ce2beea511 None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:19.499183 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014029a9-edc1-46b5-a078-e9ce2beea511 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:19.518804 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.auth.core [None req-014029a9-edc1-46b5-a078-e9ce2beea511 None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80312) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:27:19.520147 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-014029a9-edc1-46b5-a078-e9ce2beea511 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:27:19.552234 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79d1b22-5fa2-4cb1-9e23-26649deb7600 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:27:19.553256 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79d1b22-5fa2-4cb1-9e23-26649deb7600 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:19.553754 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79d1b22-5fa2-4cb1-9e23-26649deb7600 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:19.554233 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79d1b22-5fa2-4cb1-9e23-26649deb7600 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:19.555854 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79d1b22-5fa2-4cb1-9e23-26649deb7600 None admin] RBAC: Adding query filter params (group.id=eb554eeac000489bb0d030619756ec4b, role.id=983be40bdab04c478eaa90c6b6a46b14, scope.project.id=5bef204363a04d5d8ceab527b8803f6e) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:27:19.556570 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79d1b22-5fa2-4cb1-9e23-26649deb7600 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:27:19.558266 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79d1b22-5fa2-4cb1-9e23-26649deb7600 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:27:19.576988 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 206/424] 10.222.0.57 () {56 vars in 1376 bytes} [Fri Jul 1 16:27:19 2022] GET /identity/v3/role_assignments?group.id=eb554eeac000489bb0d030619756ec4b&scope.project.id=5bef204363a04d5d8ceab527b8803f6e&role.id=983be40bdab04c478eaa90c6b6a46b14 => generated 611 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:27:19.600737 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 219/425] 10.222.0.57 () {58 vars in 1023 bytes} [Fri Jul 1 16:27:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 01 16:27:19.625242 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79d1b22-5fa2-4cb1-9e23-26649deb7600 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:27:19.757724 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79d1b22-5fa2-4cb1-9e23-26649deb7600 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:27:19.762705 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:27:19.808671 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea831ce-a1fb-4744-a7db-1e412e729905 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:27:19.809394 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea831ce-a1fb-4744-a7db-1e412e729905 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:19.809674 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea831ce-a1fb-4744-a7db-1e412e729905 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:19.809950 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea831ce-a1fb-4744-a7db-1e412e729905 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:19.830604 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea831ce-a1fb-4744-a7db-1e412e729905 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 01 16:27:19.839189 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application [None req-dea831ce-a1fb-4744-a7db-1e412e729905 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 01 16:27:19.839189 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 16:27:19.839189 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 16:27:19.839189 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 16:27:19.839189 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 16:27:19.839189 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 16:27:19.839189 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 16:27:19.839189 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 16:27:19.839189 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 16:27:19.839189 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 16:27:19.839189 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 16:27:19.839189 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 16:27:19.839189 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 01 16:27:19.839189 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 01 16:27:19.839189 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 01 16:27:19.839189 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 01 16:27:19.840085 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 01 16:27:19.840085 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 01 16:27:19.840085 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 01 16:27:19.840085 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 01 16:27:19.840085 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 16:27:19.840085 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 16:27:19.840085 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 16:27:19.840085 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 16:27:19.840085 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 16:27:19.840085 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 16:27:19.840085 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 16:27:19.840085 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application with Lock( Jul 01 16:27:19.840085 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 16:27:19.840085 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._enter() Jul 01 16:27:19.840085 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 16:27:19.840085 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 16:27:19.840892 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 16:27:19.840892 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.creator() Jul 01 16:27:19.840892 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 16:27:19.840892 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application created_value = creator( Jul 01 16:27:19.840892 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 01 16:27:19.840892 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 01 16:27:19.840892 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 01 16:27:19.840892 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 01 16:27:19.840892 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 01 16:27:19.840892 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 01 16:27:19.840892 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 01 16:27:19.840892 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Jul 01 16:27:19.840892 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 207/426] 10.222.0.57 () {56 vars in 1099 bytes} [Fri Jul 1 16:27:19 2022] GET /identity/v3/roles/member => generated 84 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 16:27:19.847973 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014029a9-edc1-46b5-a078-e9ce2beea511 None None] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:27:19.888577 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=100 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:27:19.947765 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac75c410-e0cd-4801-8a92-c47800c4063c None None] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:19.948100 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac75c410-e0cd-4801-8a92-c47800c4063c None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:19.948348 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac75c410-e0cd-4801-8a92-c47800c4063c None None] PATH_INFO: `/` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:19.949927 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 208/427] 10.222.0.57 () {54 vars in 926 bytes} [Fri Jul 1 16:27:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 01 16:27:19.957171 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7a2bfc-e7db-486a-8e79-5f9eea347796 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:19.957448 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7a2bfc-e7db-486a-8e79-5f9eea347796 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:19.957726 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7a2bfc-e7db-486a-8e79-5f9eea347796 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:19.974293 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014029a9-edc1-46b5-a078-e9ce2beea511 None None] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:27:19.976097 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-8a7a2bfc-e7db-486a-8e79-5f9eea347796 None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:27:19.977876 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-8a7a2bfc-e7db-486a-8e79-5f9eea347796 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:27:20.020770 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4414d2-8442-48a9-a0bb-f0f7ff7b22a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:27:20.022379 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4414d2-8442-48a9-a0bb-f0f7ff7b22a6 None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:20.022668 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4414d2-8442-48a9-a0bb-f0f7ff7b22a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:20.022948 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4414d2-8442-48a9-a0bb-f0f7ff7b22a6 None admin] PATH_INFO: `/v3/roles` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:20.023721 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f4414d2-8442-48a9-a0bb-f0f7ff7b22a6 None admin] RBAC: Adding query filter params (name=member) {{(pid=80312) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:27:20.024064 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f4414d2-8442-48a9-a0bb-f0f7ff7b22a6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:27:20.025917 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f4414d2-8442-48a9-a0bb-f0f7ff7b22a6 None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:27:20.037682 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 220/428] 10.222.0.57 () {56 vars in 1101 bytes} [Fri Jul 1 16:27:19 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:27:20.045930 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4414d2-8442-48a9-a0bb-f0f7ff7b22a6 None admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:27:20.061367 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 209/429] 10.222.0.57 () {58 vars in 1023 bytes} [Fri Jul 1 16:27:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 01 16:27:20.088615 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7a2bfc-e7db-486a-8e79-5f9eea347796 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:27:20.165261 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=100 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:27:20.176969 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4414d2-8442-48a9-a0bb-f0f7ff7b22a6 None admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:27:20.214769 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:27:20.223063 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a826c50c-242a-4995-8268-61ace2790596 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:27:20.223722 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a826c50c-242a-4995-8268-61ace2790596 None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:20.224055 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a826c50c-242a-4995-8268-61ace2790596 None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:20.224336 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a826c50c-242a-4995-8268-61ace2790596 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:20.228644 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7a2bfc-e7db-486a-8e79-5f9eea347796 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:27:20.268586 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c11ce4c-6c12-42ed-9b4e-e01d52a7da7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:27:20.269395 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c11ce4c-6c12-42ed-9b4e-e01d52a7da7c None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:20.269739 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c11ce4c-6c12-42ed-9b4e-e01d52a7da7c None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:20.270025 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c11ce4c-6c12-42ed-9b4e-e01d52a7da7c None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:20.281949 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c11ce4c-6c12-42ed-9b4e-e01d52a7da7c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 01 16:27:20.291487 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application [None req-8c11ce4c-6c12-42ed-9b4e-e01d52a7da7c None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 01 16:27:20.291487 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 16:27:20.291487 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 16:27:20.291487 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 16:27:20.291487 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 16:27:20.291487 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 16:27:20.291487 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 16:27:20.291487 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 16:27:20.291487 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 16:27:20.291487 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 16:27:20.291487 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 16:27:20.291487 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 16:27:20.291487 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 01 16:27:20.291487 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 01 16:27:20.291487 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 01 16:27:20.291487 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 01 16:27:20.292749 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 01 16:27:20.292749 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 01 16:27:20.292749 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 01 16:27:20.292749 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 01 16:27:20.292749 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 16:27:20.292749 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 16:27:20.292749 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 16:27:20.292749 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 16:27:20.292749 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 16:27:20.292749 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 16:27:20.292749 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 16:27:20.292749 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application with Lock( Jul 01 16:27:20.292749 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 16:27:20.292749 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._enter() Jul 01 16:27:20.292749 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 16:27:20.292749 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 16:27:20.293664 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 16:27:20.293664 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.creator() Jul 01 16:27:20.293664 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 16:27:20.293664 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application created_value = creator( Jul 01 16:27:20.293664 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 01 16:27:20.293664 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 01 16:27:20.293664 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 01 16:27:20.293664 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 01 16:27:20.293664 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 01 16:27:20.293664 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 01 16:27:20.293664 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 01 16:27:20.293664 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Jul 01 16:27:20.293664 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 210/430] 10.222.0.57 () {56 vars in 1114 bytes} [Fri Jul 1 16:27:20 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 01 16:27:20.302147 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c11ce4c-6c12-42ed-9b4e-e01d52a7da7c None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:27:20.310552 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a826c50c-242a-4995-8268-61ace2790596 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:27:20.312996 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a826c50c-242a-4995-8268-61ace2790596 None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:27:20.426812 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c11ce4c-6c12-42ed-9b4e-e01d52a7da7c None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:27:20.435833 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=100 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:27:20.450515 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 221/431] 10.222.0.57 () {58 vars in 1282 bytes} [Fri Jul 1 16:27:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 405 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 01 16:27:20.459133 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a826c50c-242a-4995-8268-61ace2790596 None admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:27:20.482739 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320ca5a8-1b52-4923-8742-d6a14d13d681 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:27:20.483682 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320ca5a8-1b52-4923-8742-d6a14d13d681 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:20.484125 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320ca5a8-1b52-4923-8742-d6a14d13d681 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:20.484544 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320ca5a8-1b52-4923-8742-d6a14d13d681 None admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:20.485442 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-320ca5a8-1b52-4923-8742-d6a14d13d681 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:27:20.486450 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-320ca5a8-1b52-4923-8742-d6a14d13d681 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:27:20.488511 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-320ca5a8-1b52-4923-8742-d6a14d13d681 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:27:20.498562 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 211/432] 10.222.0.57 () {56 vars in 1111 bytes} [Fri Jul 1 16:27:20 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:27:20.505577 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320ca5a8-1b52-4923-8742-d6a14d13d681 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:27:20.525025 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:27:20.597876 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a826c50c-242a-4995-8268-61ace2790596 None admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:27:20.624843 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320ca5a8-1b52-4923-8742-d6a14d13d681 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:27:20.641435 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f048881-e542-424f-ab21-f5240883846a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:27:20.642611 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f048881-e542-424f-ab21-f5240883846a None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:20.642899 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f048881-e542-424f-ab21-f5240883846a None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:20.643205 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f048881-e542-424f-ab21-f5240883846a None admin] PATH_INFO: `/v3/projects/63f84f2aa0bf4707bd9551422f0cae3e` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:20.644746 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f048881-e542-424f-ab21-f5240883846a None admin] RBAC: Authorizing `identity:get_project(project_id=63f84f2aa0bf4707bd9551422f0cae3e)` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:27:20.646741 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f048881-e542-424f-ab21-f5240883846a None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:27:20.649944 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 222/433] 10.222.0.57 () {56 vars in 1186 bytes} [Fri Jul 1 16:27:20 2022] GET /identity/v3/projects/63f84f2aa0bf4707bd9551422f0cae3e => generated 302 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:27:20.655088 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f048881-e542-424f-ab21-f5240883846a None admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:27:20.666230 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785f53ca-acd8-4cb8-86dc-41023e333b11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:27:20.666915 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785f53ca-acd8-4cb8-86dc-41023e333b11 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:20.667201 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785f53ca-acd8-4cb8-86dc-41023e333b11 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:20.667479 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785f53ca-acd8-4cb8-86dc-41023e333b11 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:20.730068 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=100 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:27:20.741941 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785f53ca-acd8-4cb8-86dc-41023e333b11 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:27:20.743927 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785f53ca-acd8-4cb8-86dc-41023e333b11 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:27:20.760104 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:27:20.773231 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f048881-e542-424f-ab21-f5240883846a None admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:27:20.807832 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24370de2-788e-4e7d-8fd6-03ba32a73ac0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:27:20.808687 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24370de2-788e-4e7d-8fd6-03ba32a73ac0 None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:20.808979 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24370de2-788e-4e7d-8fd6-03ba32a73ac0 None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:20.809245 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24370de2-788e-4e7d-8fd6-03ba32a73ac0 None admin] PATH_INFO: `/v3/groups/eb554eeac000489bb0d030619756ec4b` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:20.810582 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24370de2-788e-4e7d-8fd6-03ba32a73ac0 None admin] RBAC: Authorizing `identity:get_group(group_id=eb554eeac000489bb0d030619756ec4b)` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:27:20.812034 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24370de2-788e-4e7d-8fd6-03ba32a73ac0 None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:27:20.818519 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 223/434] 10.222.0.57 () {56 vars in 1180 bytes} [Fri Jul 1 16:27:20 2022] GET /identity/v3/groups/eb554eeac000489bb0d030619756ec4b => generated 226 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 01 16:27:20.818519 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0aeea79-0572-49e8-b79d-7f18dbe109e1 None None] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:20.818519 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0aeea79-0572-49e8-b79d-7f18dbe109e1 None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:20.818519 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0aeea79-0572-49e8-b79d-7f18dbe109e1 None None] PATH_INFO: `/` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:20.818519 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 224/435] 10.222.0.57 () {54 vars in 926 bytes} [Fri Jul 1 16:27:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 16:27:20.822107 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0aeea79-0572-49e8-b79d-7f18dbe109e1 None None] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:27:20.862489 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 212/436] 10.222.0.57 () {58 vars in 1282 bytes} [Fri Jul 1 16:27:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 01 16:27:20.867884 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69e2347-faa1-4b99-a4d3-6f2d44fcb86c None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:20.868167 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69e2347-faa1-4b99-a4d3-6f2d44fcb86c None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:20.868441 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69e2347-faa1-4b99-a4d3-6f2d44fcb86c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:20.885281 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-e69e2347-faa1-4b99-a4d3-6f2d44fcb86c None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:27:20.886509 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-e69e2347-faa1-4b99-a4d3-6f2d44fcb86c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:27:20.942401 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0aeea79-0572-49e8-b79d-7f18dbe109e1 None None] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:27:20.955431 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 213/437] 10.222.0.57 () {58 vars in 1023 bytes} [Fri Jul 1 16:27:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 01 16:27:20.956262 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69e2347-faa1-4b99-a4d3-6f2d44fcb86c None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:27:20.988241 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=100 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:27:21.000635 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9010467f-a9ad-465a-86e4-14c815ae5b4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:27:21.001618 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9010467f-a9ad-465a-86e4-14c815ae5b4c None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:21.002590 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9010467f-a9ad-465a-86e4-14c815ae5b4c None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:21.003011 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9010467f-a9ad-465a-86e4-14c815ae5b4c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:21.004426 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9010467f-a9ad-465a-86e4-14c815ae5b4c None admin] RBAC: Adding query filter params (group.id=eb554eeac000489bb0d030619756ec4b, role.id=983be40bdab04c478eaa90c6b6a46b14, scope.project.id=63f84f2aa0bf4707bd9551422f0cae3e) {{(pid=80312) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:27:21.004885 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9010467f-a9ad-465a-86e4-14c815ae5b4c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:27:21.007200 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9010467f-a9ad-465a-86e4-14c815ae5b4c None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:27:21.008306 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:27:21.015462 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 225/438] 10.222.0.57 () {56 vars in 1376 bytes} [Fri Jul 1 16:27:20 2022] GET /identity/v3/role_assignments?group.id=eb554eeac000489bb0d030619756ec4b&scope.project.id=63f84f2aa0bf4707bd9551422f0cae3e&role.id=983be40bdab04c478eaa90c6b6a46b14 => generated 611 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:27:21.015462 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9010467f-a9ad-465a-86e4-14c815ae5b4c None admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:27:21.093928 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69e2347-faa1-4b99-a4d3-6f2d44fcb86c None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:27:21.142395 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa22714-9d75-4b89-8e8b-6417d63280d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:27:21.143487 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa22714-9d75-4b89-8e8b-6417d63280d4 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:21.143775 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa22714-9d75-4b89-8e8b-6417d63280d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:21.144222 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa22714-9d75-4b89-8e8b-6417d63280d4 None admin] PATH_INFO: `/v3/projects/5bef204363a04d5d8ceab527b8803f6e` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:21.145825 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa22714-9d75-4b89-8e8b-6417d63280d4 None admin] RBAC: Authorizing `identity:get_project(project_id=5bef204363a04d5d8ceab527b8803f6e)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:27:21.147359 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa22714-9d75-4b89-8e8b-6417d63280d4 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:27:21.151032 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 214/439] 10.222.0.57 () {56 vars in 1186 bytes} [Fri Jul 1 16:27:20 2022] GET /identity/v3/projects/5bef204363a04d5d8ceab527b8803f6e => generated 298 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:27:21.156677 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa22714-9d75-4b89-8e8b-6417d63280d4 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:27:21.169200 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9010467f-a9ad-465a-86e4-14c815ae5b4c None admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:27:21.224477 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bdf15ab-6150-4b3d-97b2-92710adc407e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:27:21.225319 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bdf15ab-6150-4b3d-97b2-92710adc407e None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:21.225629 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bdf15ab-6150-4b3d-97b2-92710adc407e None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:21.225997 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bdf15ab-6150-4b3d-97b2-92710adc407e None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:21.243454 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bdf15ab-6150-4b3d-97b2-92710adc407e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 01 16:27:21.255137 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application [None req-8bdf15ab-6150-4b3d-97b2-92710adc407e None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 01 16:27:21.255137 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 16:27:21.255137 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 16:27:21.255137 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 16:27:21.255137 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 16:27:21.255137 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 16:27:21.255137 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 16:27:21.255137 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 16:27:21.255137 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 16:27:21.255137 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 16:27:21.255137 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 16:27:21.255137 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 16:27:21.255137 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 01 16:27:21.255137 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 01 16:27:21.255137 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 01 16:27:21.255137 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 01 16:27:21.256058 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 01 16:27:21.256058 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 01 16:27:21.256058 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 01 16:27:21.256058 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 01 16:27:21.256058 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 16:27:21.256058 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 16:27:21.256058 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 16:27:21.256058 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 16:27:21.256058 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 16:27:21.256058 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 16:27:21.256058 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 16:27:21.256058 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application with Lock( Jul 01 16:27:21.256058 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 16:27:21.256058 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self._enter() Jul 01 16:27:21.256058 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 16:27:21.256058 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 16:27:21.257036 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 16:27:21.257036 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.creator() Jul 01 16:27:21.257036 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 16:27:21.257036 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application created_value = creator( Jul 01 16:27:21.257036 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 01 16:27:21.257036 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 01 16:27:21.257036 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 01 16:27:21.257036 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 01 16:27:21.257036 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 01 16:27:21.257036 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 01 16:27:21.257036 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 01 16:27:21.257036 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application Jul 01 16:27:21.257036 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 226/440] 10.222.0.57 () {56 vars in 1114 bytes} [Fri Jul 1 16:27:21 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 16:27:21.262858 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bdf15ab-6150-4b3d-97b2-92710adc407e None admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:27:21.290135 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa22714-9d75-4b89-8e8b-6417d63280d4 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:27:21.304003 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:27:21.322541 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=100 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:27:21.330451 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f39c8ab-fa1d-4e21-acf9-ff59034ea885 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:27:21.331307 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f39c8ab-fa1d-4e21-acf9-ff59034ea885 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:21.331591 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f39c8ab-fa1d-4e21-acf9-ff59034ea885 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:21.331872 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f39c8ab-fa1d-4e21-acf9-ff59034ea885 None admin] PATH_INFO: `/v3/groups/eb554eeac000489bb0d030619756ec4b` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:21.333052 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f39c8ab-fa1d-4e21-acf9-ff59034ea885 None admin] RBAC: Authorizing `identity:get_group(group_id=eb554eeac000489bb0d030619756ec4b)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:27:21.334521 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f39c8ab-fa1d-4e21-acf9-ff59034ea885 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:27:21.341000 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 215/441] 10.222.0.57 () {56 vars in 1180 bytes} [Fri Jul 1 16:27:21 2022] GET /identity/v3/groups/eb554eeac000489bb0d030619756ec4b => generated 226 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:27:21.341823 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f39c8ab-fa1d-4e21-acf9-ff59034ea885 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:27:21.376384 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bdf15ab-6150-4b3d-97b2-92710adc407e None admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:27:21.413320 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73f7eb8-c0e5-43f6-9af3-0e701500c1b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:27:21.413912 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73f7eb8-c0e5-43f6-9af3-0e701500c1b1 None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:21.414215 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73f7eb8-c0e5-43f6-9af3-0e701500c1b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:21.415030 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73f7eb8-c0e5-43f6-9af3-0e701500c1b1 None admin] PATH_INFO: `/v3/roles` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:21.415884 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d73f7eb8-c0e5-43f6-9af3-0e701500c1b1 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80312) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:27:21.416274 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d73f7eb8-c0e5-43f6-9af3-0e701500c1b1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:27:21.417973 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d73f7eb8-c0e5-43f6-9af3-0e701500c1b1 None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:27:21.426389 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 227/442] 10.222.0.57 () {56 vars in 1111 bytes} [Fri Jul 1 16:27:21 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 01 16:27:21.432811 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73f7eb8-c0e5-43f6-9af3-0e701500c1b1 None admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:27:21.450367 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f39c8ab-fa1d-4e21-acf9-ff59034ea885 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:27:21.485155 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1032904-cfa0-4d36-8d6f-c670666f244d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:27:21.485807 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1032904-cfa0-4d36-8d6f-c670666f244d None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:21.486085 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1032904-cfa0-4d36-8d6f-c670666f244d None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:21.486384 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1032904-cfa0-4d36-8d6f-c670666f244d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:21.487212 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1032904-cfa0-4d36-8d6f-c670666f244d None admin] RBAC: Adding query filter params (group.id=eb554eeac000489bb0d030619756ec4b, role.id=a8e5c73da81d4f0eb7b92c7058e500fe, scope.project.id=5bef204363a04d5d8ceab527b8803f6e) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:27:21.487523 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1032904-cfa0-4d36-8d6f-c670666f244d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:27:21.488966 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1032904-cfa0-4d36-8d6f-c670666f244d None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:27:21.495567 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 216/443] 10.222.0.57 () {56 vars in 1376 bytes} [Fri Jul 1 16:27:21 2022] GET /identity/v3/role_assignments?group.id=eb554eeac000489bb0d030619756ec4b&scope.project.id=5bef204363a04d5d8ceab527b8803f6e&role.id=a8e5c73da81d4f0eb7b92c7058e500fe => generated 611 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 01 16:27:21.496776 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c6eb89-7a89-4418-813c-08ac0b3bedfd None None] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:21.497060 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c6eb89-7a89-4418-813c-08ac0b3bedfd None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:21.497336 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c6eb89-7a89-4418-813c-08ac0b3bedfd None None] PATH_INFO: `/` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:21.498603 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 217/444] 10.222.0.57 () {54 vars in 926 bytes} [Fri Jul 1 16:27:21 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 16:27:21.504335 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f8dbb6-218c-4dea-b16c-4248229ae94b None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:21.504607 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f8dbb6-218c-4dea-b16c-4248229ae94b None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:21.505083 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f8dbb6-218c-4dea-b16c-4248229ae94b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:21.525868 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-62f8dbb6-218c-4dea-b16c-4248229ae94b None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:27:21.527799 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-62f8dbb6-218c-4dea-b16c-4248229ae94b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:27:21.556590 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73f7eb8-c0e5-43f6-9af3-0e701500c1b1 None admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:27:21.579362 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=100 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:27:21.586123 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:27:21.610565 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2977e1b7-283f-4555-9386-82f1e5aa565b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:27:21.611839 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2977e1b7-283f-4555-9386-82f1e5aa565b None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:21.612342 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2977e1b7-283f-4555-9386-82f1e5aa565b None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:21.612802 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2977e1b7-283f-4555-9386-82f1e5aa565b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:21.628809 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 218/445] 10.222.0.57 () {58 vars in 1023 bytes} [Fri Jul 1 16:27:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 01 16:27:21.656334 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f8dbb6-218c-4dea-b16c-4248229ae94b None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:27:21.696862 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2977e1b7-283f-4555-9386-82f1e5aa565b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:27:21.698251 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2977e1b7-283f-4555-9386-82f1e5aa565b None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:27:21.789418 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f8dbb6-218c-4dea-b16c-4248229ae94b None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:27:21.820741 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 228/446] 10.222.0.57 () {58 vars in 1282 bytes} [Fri Jul 1 16:27:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 388 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 01 16:27:21.827179 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2977e1b7-283f-4555-9386-82f1e5aa565b None admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:27:21.835742 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163b8c6d-7d76-4f4c-b9ec-2369964e511b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:27:21.837074 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163b8c6d-7d76-4f4c-b9ec-2369964e511b None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:21.837357 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163b8c6d-7d76-4f4c-b9ec-2369964e511b None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:21.837632 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163b8c6d-7d76-4f4c-b9ec-2369964e511b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:21.854333 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-163b8c6d-7d76-4f4c-b9ec-2369964e511b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 01 16:27:21.857393 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=100 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:27:21.862201 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application [None req-163b8c6d-7d76-4f4c-b9ec-2369964e511b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 01 16:27:21.862201 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 16:27:21.862201 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 16:27:21.862201 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 16:27:21.862201 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 16:27:21.862201 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 16:27:21.862201 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 16:27:21.862201 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 16:27:21.862201 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 16:27:21.862201 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 16:27:21.862201 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 16:27:21.862201 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 16:27:21.862201 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 01 16:27:21.862201 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 01 16:27:21.862201 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 01 16:27:21.862201 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 01 16:27:21.863168 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 01 16:27:21.863168 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 01 16:27:21.863168 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 01 16:27:21.863168 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 01 16:27:21.863168 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 16:27:21.863168 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 16:27:21.863168 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 16:27:21.863168 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 16:27:21.863168 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 16:27:21.863168 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 16:27:21.863168 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 16:27:21.863168 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application with Lock( Jul 01 16:27:21.863168 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 16:27:21.863168 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._enter() Jul 01 16:27:21.863168 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 16:27:21.863168 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 16:27:21.864000 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 16:27:21.864000 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.creator() Jul 01 16:27:21.864000 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 16:27:21.864000 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application created_value = creator( Jul 01 16:27:21.864000 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 01 16:27:21.864000 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 01 16:27:21.864000 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 01 16:27:21.864000 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 01 16:27:21.864000 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 01 16:27:21.864000 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 01 16:27:21.864000 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 01 16:27:21.864000 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Jul 01 16:27:21.864596 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 219/447] 10.222.0.57 () {56 vars in 1096 bytes} [Fri Jul 1 16:27:21 2022] GET /identity/v3/roles/admin => generated 83 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 01 16:27:21.871288 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163b8c6d-7d76-4f4c-b9ec-2369964e511b None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:27:21.911794 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:27:21.968900 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2977e1b7-283f-4555-9386-82f1e5aa565b None admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:27:22.014708 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163b8c6d-7d76-4f4c-b9ec-2369964e511b None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:27:22.023115 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74253f2e-8cb0-4b92-9081-e6540f69dfc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:27:22.024865 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74253f2e-8cb0-4b92-9081-e6540f69dfc7 None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:22.025311 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74253f2e-8cb0-4b92-9081-e6540f69dfc7 None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:22.026229 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74253f2e-8cb0-4b92-9081-e6540f69dfc7 None admin] PATH_INFO: `/v3/projects/63f84f2aa0bf4707bd9551422f0cae3e` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:22.028378 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74253f2e-8cb0-4b92-9081-e6540f69dfc7 None admin] RBAC: Authorizing `identity:get_project(project_id=63f84f2aa0bf4707bd9551422f0cae3e)` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:27:22.029975 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74253f2e-8cb0-4b92-9081-e6540f69dfc7 None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:27:22.031843 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 229/448] 10.222.0.57 () {56 vars in 1186 bytes} [Fri Jul 1 16:27:21 2022] GET /identity/v3/projects/63f84f2aa0bf4707bd9551422f0cae3e => generated 302 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 01 16:27:22.038789 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74253f2e-8cb0-4b92-9081-e6540f69dfc7 None admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:27:22.060374 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fdeb5b7-55b7-40fa-ab9d-4a20e905814a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:27:22.061073 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdeb5b7-55b7-40fa-ab9d-4a20e905814a None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:22.061332 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdeb5b7-55b7-40fa-ab9d-4a20e905814a None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:22.061613 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdeb5b7-55b7-40fa-ab9d-4a20e905814a None admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:22.062429 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fdeb5b7-55b7-40fa-ab9d-4a20e905814a None admin] RBAC: Adding query filter params (name=admin) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:27:22.062746 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fdeb5b7-55b7-40fa-ab9d-4a20e905814a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:27:22.064127 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fdeb5b7-55b7-40fa-ab9d-4a20e905814a None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:27:22.074278 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 220/449] 10.222.0.57 () {56 vars in 1099 bytes} [Fri Jul 1 16:27:21 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:27:22.081652 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fdeb5b7-55b7-40fa-ab9d-4a20e905814a None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:27:22.138655 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=100 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:27:22.159286 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74253f2e-8cb0-4b92-9081-e6540f69dfc7 None admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:27:22.196586 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:27:22.199156 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251fec5a-422d-46cd-8d52-e08a10443e0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:27:22.200530 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251fec5a-422d-46cd-8d52-e08a10443e0d None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:22.200998 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251fec5a-422d-46cd-8d52-e08a10443e0d None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:22.200998 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251fec5a-422d-46cd-8d52-e08a10443e0d None admin] PATH_INFO: `/v3/groups/eb554eeac000489bb0d030619756ec4b` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:22.202525 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-251fec5a-422d-46cd-8d52-e08a10443e0d None admin] RBAC: Authorizing `identity:get_group(group_id=eb554eeac000489bb0d030619756ec4b)` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:27:22.202909 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fdeb5b7-55b7-40fa-ab9d-4a20e905814a None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:27:22.204025 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-251fec5a-422d-46cd-8d52-e08a10443e0d None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:27:22.205939 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 230/450] 10.222.0.57 () {56 vars in 1180 bytes} [Fri Jul 1 16:27:22 2022] GET /identity/v3/groups/eb554eeac000489bb0d030619756ec4b => generated 226 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 01 16:27:22.212448 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251fec5a-422d-46cd-8d52-e08a10443e0d None admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:27:22.241924 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f0072f-7be6-4606-93e5-e4d3123e3553 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:27:22.242611 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f0072f-7be6-4606-93e5-e4d3123e3553 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:22.242924 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f0072f-7be6-4606-93e5-e4d3123e3553 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:22.243208 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f0072f-7be6-4606-93e5-e4d3123e3553 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:22.328767 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251fec5a-422d-46cd-8d52-e08a10443e0d None admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:27:22.334609 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f0072f-7be6-4606-93e5-e4d3123e3553 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:27:22.336153 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f0072f-7be6-4606-93e5-e4d3123e3553 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:27:22.372649 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d57164-2f2d-4e97-8633-bb5b8bee1a6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:27:22.373306 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d57164-2f2d-4e97-8633-bb5b8bee1a6b None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:22.373586 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d57164-2f2d-4e97-8633-bb5b8bee1a6b None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:22.373861 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d57164-2f2d-4e97-8633-bb5b8bee1a6b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:22.374733 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d57164-2f2d-4e97-8633-bb5b8bee1a6b None admin] RBAC: Adding query filter params (group.id=eb554eeac000489bb0d030619756ec4b, role.id=a8e5c73da81d4f0eb7b92c7058e500fe, scope.project.id=63f84f2aa0bf4707bd9551422f0cae3e) {{(pid=80312) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:27:22.375050 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d57164-2f2d-4e97-8633-bb5b8bee1a6b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:27:22.377209 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d57164-2f2d-4e97-8633-bb5b8bee1a6b None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:27:22.384081 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 231/451] 10.222.0.57 () {56 vars in 1376 bytes} [Fri Jul 1 16:27:22 2022] GET /identity/v3/role_assignments?group.id=eb554eeac000489bb0d030619756ec4b&scope.project.id=63f84f2aa0bf4707bd9551422f0cae3e&role.id=a8e5c73da81d4f0eb7b92c7058e500fe => generated 611 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 01 16:27:22.448253 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:27:22.457420 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 221/452] 10.222.0.57 () {58 vars in 1282 bytes} [Fri Jul 1 16:27:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 01 16:27:22.463303 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d57164-2f2d-4e97-8633-bb5b8bee1a6b None admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:27:22.482047 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=100 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:27:22.582151 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d57164-2f2d-4e97-8633-bb5b8bee1a6b None admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:27:22.624434 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33e9835-a7b3-465c-b124-ece80e534a6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:27:22.625301 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33e9835-a7b3-465c-b124-ece80e534a6e None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:22.625583 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33e9835-a7b3-465c-b124-ece80e534a6e None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:22.625854 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33e9835-a7b3-465c-b124-ece80e534a6e None admin] PATH_INFO: `/v3/projects/62a34c02685249b399e8facc39cc11f8` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:22.627517 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f33e9835-a7b3-465c-b124-ece80e534a6e None admin] RBAC: Authorizing `identity:get_project(project_id=62a34c02685249b399e8facc39cc11f8)` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:27:22.629954 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f33e9835-a7b3-465c-b124-ece80e534a6e None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:27:22.632501 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 232/453] 10.222.0.57 () {56 vars in 1186 bytes} [Fri Jul 1 16:27:22 2022] GET /identity/v3/projects/62a34c02685249b399e8facc39cc11f8 => generated 344 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:27:22.639064 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f0072f-7be6-4606-93e5-e4d3123e3553 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:27:22.751741 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f0072f-7be6-4606-93e5-e4d3123e3553 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:27:22.794575 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16af64d8-0064-443e-802b-a4ac4b2d2186 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:27:22.795820 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16af64d8-0064-443e-802b-a4ac4b2d2186 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:22.796371 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16af64d8-0064-443e-802b-a4ac4b2d2186 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:22.796371 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16af64d8-0064-443e-802b-a4ac4b2d2186 None admin] PATH_INFO: `/v3/groups/1b0582f065d1420ea828d3d4082fdc87` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:22.797570 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16af64d8-0064-443e-802b-a4ac4b2d2186 None admin] RBAC: Authorizing `identity:get_group(group_id=1b0582f065d1420ea828d3d4082fdc87)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:27:22.799125 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16af64d8-0064-443e-802b-a4ac4b2d2186 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:27:22.800783 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 222/454] 10.222.0.57 () {56 vars in 1180 bytes} [Fri Jul 1 16:27:22 2022] GET /identity/v3/groups/1b0582f065d1420ea828d3d4082fdc87 => generated 229 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:27:22.806818 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33e9835-a7b3-465c-b124-ece80e534a6e None admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:27:22.888197 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=100 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:27:22.906837 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33e9835-a7b3-465c-b124-ece80e534a6e None admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:27:22.938674 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4f1d82-0484-4d31-ba46-e33d22f11d21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:27:22.939378 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4f1d82-0484-4d31-ba46-e33d22f11d21 None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:22.939660 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4f1d82-0484-4d31-ba46-e33d22f11d21 None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:22.939931 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4f1d82-0484-4d31-ba46-e33d22f11d21 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:22.940752 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e4f1d82-0484-4d31-ba46-e33d22f11d21 None admin] RBAC: Adding query filter params (group.id=1b0582f065d1420ea828d3d4082fdc87, role.id=84ee739146934c1ab32c501ff8093644, scope.project.id=62a34c02685249b399e8facc39cc11f8) {{(pid=80312) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:27:22.941061 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e4f1d82-0484-4d31-ba46-e33d22f11d21 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:27:22.942511 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e4f1d82-0484-4d31-ba46-e33d22f11d21 None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:27:22.948799 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 233/455] 10.222.0.57 () {56 vars in 1376 bytes} [Fri Jul 1 16:27:22 2022] GET /identity/v3/role_assignments?group.id=1b0582f065d1420ea828d3d4082fdc87&scope.project.id=62a34c02685249b399e8facc39cc11f8&role.id=84ee739146934c1ab32c501ff8093644 => generated 611 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 01 16:27:25.369391 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d8e1e1-37e2-40de-b509-c0e63b7a9a50 None None] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:25.369910 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d8e1e1-37e2-40de-b509-c0e63b7a9a50 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:25.369986 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d8e1e1-37e2-40de-b509-c0e63b7a9a50 None None] PATH_INFO: `/` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:25.371620 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 223/456] 10.222.0.57 () {54 vars in 926 bytes} [Fri Jul 1 16:27:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 16:27:25.376782 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb27327-1109-47c5-ac0d-cff60379b454 None None] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:25.377213 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb27327-1109-47c5-ac0d-cff60379b454 None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:25.377358 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb27327-1109-47c5-ac0d-cff60379b454 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:25.403402 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.auth.core [None req-fdb27327-1109-47c5-ac0d-cff60379b454 None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80312) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:27:25.404605 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-fdb27327-1109-47c5-ac0d-cff60379b454 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:27:25.493103 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 234/457] 10.222.0.57 () {58 vars in 1023 bytes} [Fri Jul 1 16:27:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 01 16:27:25.529729 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d8e1e1-37e2-40de-b509-c0e63b7a9a50 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:27:25.608102 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:27:25.665974 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d8e1e1-37e2-40de-b509-c0e63b7a9a50 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:27:25.709897 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02cd57c7-7434-4154-ba59-88b8819317e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:27:25.710736 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cd57c7-7434-4154-ba59-88b8819317e8 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:25.711044 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cd57c7-7434-4154-ba59-88b8819317e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:25.711417 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cd57c7-7434-4154-ba59-88b8819317e8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:25.724595 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02cd57c7-7434-4154-ba59-88b8819317e8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:27:25.726050 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02cd57c7-7434-4154-ba59-88b8819317e8 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:27:25.755408 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application [None req-02cd57c7-7434-4154-ba59-88b8819317e8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 16:27:25.755408 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 16:27:25.755408 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 16:27:25.755408 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 16:27:25.755408 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 16:27:25.755408 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 16:27:25.755408 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 16:27:25.755408 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 16:27:25.755408 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 16:27:25.755408 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 16:27:25.755408 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 16:27:25.755408 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 16:27:25.755408 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 01 16:27:25.755408 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 01 16:27:25.755408 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 01 16:27:25.756335 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 01 16:27:25.756335 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 16:27:25.756335 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 16:27:25.756335 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 16:27:25.756335 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 16:27:25.756335 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 16:27:25.756335 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 16:27:25.756335 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 16:27:25.756335 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application with Lock( Jul 01 16:27:25.756335 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 16:27:25.756335 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._enter() Jul 01 16:27:25.756335 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 16:27:25.756335 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 16:27:25.756335 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 16:27:25.756335 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.creator() Jul 01 16:27:25.756335 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 16:27:25.756335 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application created_value = creator( Jul 01 16:27:25.757200 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 01 16:27:25.757200 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 01 16:27:25.757200 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 16:27:25.757200 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Jul 01 16:27:25.757200 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 224/458] 10.222.0.57 () {56 vars in 1108 bytes} [Fri Jul 1 16:27:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 16:27:25.764476 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb27327-1109-47c5-ac0d-cff60379b454 None None] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:27:25.777436 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab9fdad-8307-4fc8-a70a-f457dcd385fd None None] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:25.777756 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab9fdad-8307-4fc8-a70a-f457dcd385fd None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:25.777993 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab9fdad-8307-4fc8-a70a-f457dcd385fd None None] PATH_INFO: `/` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:25.780675 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 225/459] 10.222.0.57 () {54 vars in 926 bytes} [Fri Jul 1 16:27:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 16:27:25.784661 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3437b289-0c96-4338-83e8-a47621709e24 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:25.785143 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3437b289-0c96-4338-83e8-a47621709e24 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:25.785430 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3437b289-0c96-4338-83e8-a47621709e24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:25.816677 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-3437b289-0c96-4338-83e8-a47621709e24 None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:27:25.818148 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-3437b289-0c96-4338-83e8-a47621709e24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:27:25.884743 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=100 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:27:25.910541 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb27327-1109-47c5-ac0d-cff60379b454 None None] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:27:25.915427 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 226/460] 10.222.0.57 () {58 vars in 1023 bytes} [Fri Jul 1 16:27:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 01 16:27:25.917027 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3db0e2f-e05c-4489-b53f-6d1d78a0b0df None None] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:25.917355 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3db0e2f-e05c-4489-b53f-6d1d78a0b0df None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:25.917728 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3db0e2f-e05c-4489-b53f-6d1d78a0b0df None None] PATH_INFO: `/` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:25.923228 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 227/461] 10.222.0.57 () {54 vars in 926 bytes} [Fri Jul 1 16:27:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 16:27:25.924856 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2259fce5-583e-4d3b-b302-aaf2a2cddd7f None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:25.925145 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2259fce5-583e-4d3b-b302-aaf2a2cddd7f None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:25.925424 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2259fce5-583e-4d3b-b302-aaf2a2cddd7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:25.958268 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f725a956-9cbd-4397-beee-e2797ea44a38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:27:25.959365 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f725a956-9cbd-4397-beee-e2797ea44a38 None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:25.959939 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f725a956-9cbd-4397-beee-e2797ea44a38 None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:25.960421 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f725a956-9cbd-4397-beee-e2797ea44a38 None admin] PATH_INFO: `/v3/domains` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:25.961439 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f725a956-9cbd-4397-beee-e2797ea44a38 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80312) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:27:25.961563 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-2259fce5-583e-4d3b-b302-aaf2a2cddd7f None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:27:25.962141 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f725a956-9cbd-4397-beee-e2797ea44a38 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:27:25.962825 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-2259fce5-583e-4d3b-b302-aaf2a2cddd7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:27:25.964205 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f725a956-9cbd-4397-beee-e2797ea44a38 None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:27:25.980655 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 235/462] 10.222.0.57 () {56 vars in 1109 bytes} [Fri Jul 1 16:27:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:27:25.981886 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f725a956-9cbd-4397-beee-e2797ea44a38 None admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:27:26.042673 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:27:26.057920 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 228/463] 10.222.0.57 () {58 vars in 1023 bytes} [Fri Jul 1 16:27:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 01 16:27:26.057920 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2259fce5-583e-4d3b-b302-aaf2a2cddd7f None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:27:26.118386 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f725a956-9cbd-4397-beee-e2797ea44a38 None admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:27:26.165101 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2259fce5-583e-4d3b-b302-aaf2a2cddd7f None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:27:26.167585 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5527b9-0abd-450b-9cad-3053af9a61b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:27:26.169132 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5527b9-0abd-450b-9cad-3053af9a61b5 None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:26.169424 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5527b9-0abd-450b-9cad-3053af9a61b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:26.169706 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5527b9-0abd-450b-9cad-3053af9a61b5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:26.172582 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=100 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:27:26.184714 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d5527b9-0abd-450b-9cad-3053af9a61b5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:27:26.186558 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d5527b9-0abd-450b-9cad-3053af9a61b5 None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:27:26.207290 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application [None req-1d5527b9-0abd-450b-9cad-3053af9a61b5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 16:27:26.207290 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 16:27:26.207290 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 16:27:26.207290 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 16:27:26.207290 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 16:27:26.207290 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 16:27:26.207290 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 16:27:26.207290 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 16:27:26.207290 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 16:27:26.207290 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 16:27:26.207290 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 16:27:26.207290 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 16:27:26.207290 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 01 16:27:26.207290 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 01 16:27:26.207290 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 01 16:27:26.208258 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 01 16:27:26.208258 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 16:27:26.208258 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 16:27:26.208258 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 16:27:26.208258 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 16:27:26.208258 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 16:27:26.208258 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 16:27:26.208258 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 16:27:26.208258 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application with Lock( Jul 01 16:27:26.208258 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 16:27:26.208258 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self._enter() Jul 01 16:27:26.208258 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 16:27:26.208258 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 16:27:26.208258 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 16:27:26.208258 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.creator() Jul 01 16:27:26.208258 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 16:27:26.208258 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application created_value = creator( Jul 01 16:27:26.209119 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 01 16:27:26.209119 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 01 16:27:26.209119 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 16:27:26.209119 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application Jul 01 16:27:26.211275 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 236/464] 10.222.0.57 () {56 vars in 1108 bytes} [Fri Jul 1 16:27:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 16:27:26.211768 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5d3c90-633d-4cf0-bbb6-97ab5a3269fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:27:26.212628 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5d3c90-633d-4cf0-bbb6-97ab5a3269fa None admin] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:26.212874 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5527b9-0abd-450b-9cad-3053af9a61b5 None admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:27:26.212973 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5d3c90-633d-4cf0-bbb6-97ab5a3269fa None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:26.213244 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5d3c90-633d-4cf0-bbb6-97ab5a3269fa None admin] PATH_INFO: `/v3/users` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:26.213962 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de5d3c90-633d-4cf0-bbb6-97ab5a3269fa None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:27:26.215457 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de5d3c90-633d-4cf0-bbb6-97ab5a3269fa None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:27:26.244642 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 229/465] 10.222.0.57 () {60 vars in 1132 bytes} [Fri Jul 1 16:27:26 2022] POST /identity/v3/users => generated 246 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 16:27:26.245781 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5d3c90-633d-4cf0-bbb6-97ab5a3269fa None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:27:26.337028 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:27:26.413257 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5527b9-0abd-450b-9cad-3053af9a61b5 None admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:27:26.450854 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5d3c90-633d-4cf0-bbb6-97ab5a3269fa None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:27:26.457802 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b978c113-a6c8-4b72-90e2-dcd40695677b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:27:26.458645 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b978c113-a6c8-4b72-90e2-dcd40695677b None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:26.458927 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b978c113-a6c8-4b72-90e2-dcd40695677b None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:26.459201 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b978c113-a6c8-4b72-90e2-dcd40695677b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:26.471999 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b978c113-a6c8-4b72-90e2-dcd40695677b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:27:26.474233 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b978c113-a6c8-4b72-90e2-dcd40695677b None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:27:26.492991 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application [None req-b978c113-a6c8-4b72-90e2-dcd40695677b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 16:27:26.492991 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 16:27:26.492991 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 16:27:26.492991 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 16:27:26.492991 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 16:27:26.492991 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 16:27:26.492991 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 16:27:26.492991 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 16:27:26.492991 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 16:27:26.492991 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 16:27:26.492991 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 16:27:26.492991 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 16:27:26.492991 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 01 16:27:26.492991 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 01 16:27:26.492991 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 01 16:27:26.494058 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 01 16:27:26.494058 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 16:27:26.494058 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 16:27:26.494058 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 16:27:26.494058 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 16:27:26.494058 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 16:27:26.494058 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 16:27:26.494058 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 16:27:26.494058 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application with Lock( Jul 01 16:27:26.494058 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 16:27:26.494058 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self._enter() Jul 01 16:27:26.494058 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 16:27:26.494058 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 16:27:26.494058 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 16:27:26.494058 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.creator() Jul 01 16:27:26.494058 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 16:27:26.494058 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application created_value = creator( Jul 01 16:27:26.494977 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 01 16:27:26.494977 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 01 16:27:26.494977 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 16:27:26.494977 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application Jul 01 16:27:26.494977 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 237/466] 10.222.0.57 () {56 vars in 1108 bytes} [Fri Jul 1 16:27:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 282 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 16:27:26.495745 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219d4e86-4a83-43d3-a5b2-848c798e5283 None None] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:26.495745 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219d4e86-4a83-43d3-a5b2-848c798e5283 None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:26.496023 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219d4e86-4a83-43d3-a5b2-848c798e5283 None None] PATH_INFO: `/` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:26.497147 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7681115d-2da5-46ad-a911-eb0e60548ad5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:27:26.497147 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7681115d-2da5-46ad-a911-eb0e60548ad5 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:26.497423 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7681115d-2da5-46ad-a911-eb0e60548ad5 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:26.497514 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 238/467] 10.222.0.57 () {54 vars in 926 bytes} [Fri Jul 1 16:27:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 16:27:26.497648 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7681115d-2da5-46ad-a911-eb0e60548ad5 None admin] PATH_INFO: `/v3/domains` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:26.502200 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7681115d-2da5-46ad-a911-eb0e60548ad5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:27:26.502611 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7681115d-2da5-46ad-a911-eb0e60548ad5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:27:26.502716 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219d4e86-4a83-43d3-a5b2-848c798e5283 None None] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:27:26.504013 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7681115d-2da5-46ad-a911-eb0e60548ad5 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:27:26.521003 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 230/468] 10.222.0.57 () {56 vars in 1109 bytes} [Fri Jul 1 16:27:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:27:26.522191 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a1398a-45ff-4ea9-b47b-da6ff4f0c079 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:26.522516 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a1398a-45ff-4ea9-b47b-da6ff4f0c079 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:26.522796 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a1398a-45ff-4ea9-b47b-da6ff4f0c079 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:26.543193 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-a0a1398a-45ff-4ea9-b47b-da6ff4f0c079 None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:27:26.544411 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-a0a1398a-45ff-4ea9-b47b-da6ff4f0c079 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:27:26.562274 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=100 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:27:26.633576 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 231/469] 10.222.0.57 () {58 vars in 1023 bytes} [Fri Jul 1 16:27:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 01 16:27:26.634999 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a1398a-45ff-4ea9-b47b-da6ff4f0c079 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:27:26.641561 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219d4e86-4a83-43d3-a5b2-848c798e5283 None None] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:27:26.669423 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:27:26.692637 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61848dc-d5ce-4ec4-905e-f85750ea5225 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:27:26.693951 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61848dc-d5ce-4ec4-905e-f85750ea5225 None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:26.694392 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61848dc-d5ce-4ec4-905e-f85750ea5225 None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:26.694769 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61848dc-d5ce-4ec4-905e-f85750ea5225 None admin] PATH_INFO: `/v3/domains` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:26.696589 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c61848dc-d5ce-4ec4-905e-f85750ea5225 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80312) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:27:26.696996 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c61848dc-d5ce-4ec4-905e-f85750ea5225 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:27:26.699079 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c61848dc-d5ce-4ec4-905e-f85750ea5225 None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:27:26.716588 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 239/470] 10.222.0.57 () {56 vars in 1109 bytes} [Fri Jul 1 16:27:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:27:26.721073 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61848dc-d5ce-4ec4-905e-f85750ea5225 None admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:27:26.768367 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a1398a-45ff-4ea9-b47b-da6ff4f0c079 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:27:26.808123 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1b1038-70c4-4297-a201-3085acd04029 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:27:26.809202 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1b1038-70c4-4297-a201-3085acd04029 None admin] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:26.809705 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1b1038-70c4-4297-a201-3085acd04029 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:26.810503 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1b1038-70c4-4297-a201-3085acd04029 None admin] PATH_INFO: `/v3/users` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:26.811469 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1b1038-70c4-4297-a201-3085acd04029 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:27:26.813387 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1b1038-70c4-4297-a201-3085acd04029 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:27:26.840830 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 232/471] 10.222.0.57 () {60 vars in 1132 bytes} [Fri Jul 1 16:27:26 2022] POST /identity/v3/users => generated 248 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 16:27:26.842166 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1b1038-70c4-4297-a201-3085acd04029 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:27:26.866581 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=100 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:27:26.871370 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61848dc-d5ce-4ec4-905e-f85750ea5225 None admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:27:26.958312 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d12011-f041-45ee-977a-2a09eaa846e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:27:26.959173 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d12011-f041-45ee-977a-2a09eaa846e3 None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:26.959463 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d12011-f041-45ee-977a-2a09eaa846e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:26.959745 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d12011-f041-45ee-977a-2a09eaa846e3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:26.975217 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d12011-f041-45ee-977a-2a09eaa846e3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:27:26.976857 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d12011-f041-45ee-977a-2a09eaa846e3 None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:27:26.989284 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:27:27.000911 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application [None req-d0d12011-f041-45ee-977a-2a09eaa846e3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 16:27:27.000911 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 16:27:27.000911 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 16:27:27.000911 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 16:27:27.000911 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 16:27:27.000911 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 16:27:27.000911 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 16:27:27.000911 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 16:27:27.000911 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 16:27:27.000911 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 16:27:27.000911 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 16:27:27.000911 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 16:27:27.000911 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 01 16:27:27.000911 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 01 16:27:27.000911 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 01 16:27:27.001990 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 01 16:27:27.001990 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 16:27:27.001990 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 16:27:27.001990 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 16:27:27.001990 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 16:27:27.001990 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 16:27:27.001990 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 16:27:27.001990 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 16:27:27.001990 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application with Lock( Jul 01 16:27:27.001990 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 16:27:27.001990 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self._enter() Jul 01 16:27:27.001990 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 16:27:27.001990 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 16:27:27.001990 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 16:27:27.001990 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.creator() Jul 01 16:27:27.001990 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 16:27:27.001990 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application created_value = creator( Jul 01 16:27:27.003183 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 01 16:27:27.003183 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 01 16:27:27.003183 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 16:27:27.003183 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application Jul 01 16:27:27.003183 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 240/472] 10.222.0.57 () {56 vars in 1108 bytes} [Fri Jul 1 16:27:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 285 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 16:27:27.006883 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4948046e-fbb1-4eea-bf4a-5d0e6e78a868 None None] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:27.006883 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4948046e-fbb1-4eea-bf4a-5d0e6e78a868 None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:27.006883 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4948046e-fbb1-4eea-bf4a-5d0e6e78a868 None None] PATH_INFO: `/` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:27.007736 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 241/473] 10.222.0.57 () {54 vars in 926 bytes} [Fri Jul 1 16:27:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 16:27:27.008964 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4948046e-fbb1-4eea-bf4a-5d0e6e78a868 None None] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:27:27.017668 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1b1038-70c4-4297-a201-3085acd04029 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:27:27.067204 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847d4d1c-e638-4af9-9fbb-d4c129e03c49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:27:27.067893 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847d4d1c-e638-4af9-9fbb-d4c129e03c49 None admin] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:27.068187 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847d4d1c-e638-4af9-9fbb-d4c129e03c49 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:27.068475 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847d4d1c-e638-4af9-9fbb-d4c129e03c49 None admin] PATH_INFO: `/v3/users` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:27.069203 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-847d4d1c-e638-4af9-9fbb-d4c129e03c49 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:27:27.071353 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-847d4d1c-e638-4af9-9fbb-d4c129e03c49 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:27:27.099610 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 233/474] 10.222.0.57 () {60 vars in 1132 bytes} [Fri Jul 1 16:27:26 2022] POST /identity/v3/users => generated 248 bytes in 258 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 16:27:27.100966 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c56632-c48b-4208-b721-ea8b60540349 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:27.101251 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c56632-c48b-4208-b721-ea8b60540349 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:27.101566 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c56632-c48b-4208-b721-ea8b60540349 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:27.122024 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-18c56632-c48b-4208-b721-ea8b60540349 None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:27:27.123264 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-18c56632-c48b-4208-b721-ea8b60540349 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:27:27.148916 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4948046e-fbb1-4eea-bf4a-5d0e6e78a868 None None] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:27:27.207409 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e3941e-a649-496a-a74f-18b6c618dc1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:27:27.208932 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e3941e-a649-496a-a74f-18b6c618dc1e None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:27.209441 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e3941e-a649-496a-a74f-18b6c618dc1e None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:27.210246 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e3941e-a649-496a-a74f-18b6c618dc1e None admin] PATH_INFO: `/v3/domains` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:27.212051 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e3941e-a649-496a-a74f-18b6c618dc1e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80312) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:27:27.212568 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e3941e-a649-496a-a74f-18b6c618dc1e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:27:27.215269 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e3941e-a649-496a-a74f-18b6c618dc1e None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:27:27.222169 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 234/475] 10.222.0.57 () {58 vars in 1023 bytes} [Fri Jul 1 16:27:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 01 16:27:27.232600 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 242/476] 10.222.0.57 () {56 vars in 1109 bytes} [Fri Jul 1 16:27:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:27:27.238065 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c56632-c48b-4208-b721-ea8b60540349 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:27:27.249314 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e3941e-a649-496a-a74f-18b6c618dc1e None admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:27:27.274173 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=100 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:27:27.320821 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:27:27.373447 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c56632-c48b-4208-b721-ea8b60540349 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:27:27.397582 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e3941e-a649-496a-a74f-18b6c618dc1e None admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:27:27.416218 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292b8612-2c06-4639-924b-8b057a9c2470 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:27:27.417244 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292b8612-2c06-4639-924b-8b057a9c2470 None admin] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:27.417530 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292b8612-2c06-4639-924b-8b057a9c2470 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:27.417808 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292b8612-2c06-4639-924b-8b057a9c2470 None admin] PATH_INFO: `/v3/users` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:27.418559 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-292b8612-2c06-4639-924b-8b057a9c2470 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:27:27.420022 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-292b8612-2c06-4639-924b-8b057a9c2470 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:27:27.443703 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a84f9a-e86c-4065-8e31-2387614a88f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:27:27.445777 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a84f9a-e86c-4065-8e31-2387614a88f5 None admin] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:27.445944 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 235/477] 10.222.0.57 () {60 vars in 1132 bytes} [Fri Jul 1 16:27:27 2022] POST /identity/v3/users => generated 249 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 16:27:27.446691 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a84f9a-e86c-4065-8e31-2387614a88f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:27.447235 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a84f9a-e86c-4065-8e31-2387614a88f5 None admin] PATH_INFO: `/v3/roles` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:27.448303 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a84f9a-e86c-4065-8e31-2387614a88f5 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:27:27.450065 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a84f9a-e86c-4065-8e31-2387614a88f5 None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:27:27.460879 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.sql.core [None req-f8a84f9a-e86c-4065-8e31-2387614a88f5 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 01 16:27:27.460879 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jul 01 16:27:27.460879 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [parameters: {'id': '0b0b9337d5aa4f8386deabe7c3390e2f', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jul 01 16:27:27.460879 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80312) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 01 16:27:27.480262 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application [None req-f8a84f9a-e86c-4065-8e31-2387614a88f5 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jul 01 16:27:27.480262 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 16:27:27.480262 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 01 16:27:27.480262 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 01 16:27:27.480262 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 01 16:27:27.480262 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 01 16:27:27.480262 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 01 16:27:27.480262 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application result = self._query(query) Jul 01 16:27:27.480262 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 01 16:27:27.480262 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application conn.query(q) Jul 01 16:27:27.480262 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 01 16:27:27.480262 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 01 16:27:27.480262 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 01 16:27:27.480262 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application result.read() Jul 01 16:27:27.480262 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 01 16:27:27.481199 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 01 16:27:27.481199 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 01 16:27:27.481199 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 01 16:27:27.481199 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 01 16:27:27.481199 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 01 16:27:27.481199 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 01 16:27:27.481199 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 01 16:27:27.481199 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 01 16:27:27.481199 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application Jul 01 16:27:27.481199 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 01 16:27:27.481199 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application Jul 01 16:27:27.481199 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 16:27:27.481199 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 01 16:27:27.481199 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 01 16:27:27.481199 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Jul 01 16:27:27.481199 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return ref.to_dict() Jul 01 16:27:27.481199 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 01 16:27:27.481199 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application next(self.gen) Jul 01 16:27:27.482115 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 01 16:27:27.482115 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application yield resource Jul 01 16:27:27.482115 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 01 16:27:27.482115 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application next(self.gen) Jul 01 16:27:27.482115 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 01 16:27:27.482115 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application self.session.rollback() Jul 01 16:27:27.482115 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 01 16:27:27.482115 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application self.force_reraise() Jul 01 16:27:27.482115 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 01 16:27:27.482115 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application raise self.value Jul 01 16:27:27.482115 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 01 16:27:27.482115 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 01 16:27:27.482115 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 01 16:27:27.482115 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application session.commit() Jul 01 16:27:27.482115 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 01 16:27:27.482115 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 01 16:27:27.482115 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 01 16:27:27.483110 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application self._prepare_impl() Jul 01 16:27:27.483110 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 01 16:27:27.483110 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application self.session.flush() Jul 01 16:27:27.483110 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 01 16:27:27.483110 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application self._flush(objects) Jul 01 16:27:27.483110 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 01 16:27:27.483110 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 01 16:27:27.483110 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 01 16:27:27.483110 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application compat.raise_( Jul 01 16:27:27.483110 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 01 16:27:27.483110 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application raise exception Jul 01 16:27:27.483110 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 01 16:27:27.483110 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application flush_context.execute() Jul 01 16:27:27.483110 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 01 16:27:27.483110 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application rec.execute(self) Jul 01 16:27:27.483110 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 01 16:27:27.483110 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 01 16:27:27.484953 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 01 16:27:27.484953 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 01 16:27:27.484953 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jul 01 16:27:27.484953 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application c = connection._execute_20( Jul 01 16:27:27.484953 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 01 16:27:27.484953 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 01 16:27:27.484953 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 01 16:27:27.484953 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 01 16:27:27.484953 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 01 16:27:27.484953 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 01 16:27:27.484953 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 01 16:27:27.484953 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 01 16:27:27.484953 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 01 16:27:27.484953 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 01 16:27:27.484953 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 01 16:27:27.484953 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application raise exception Jul 01 16:27:27.486987 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 01 16:27:27.486987 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 01 16:27:27.486987 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 01 16:27:27.486987 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 01 16:27:27.486987 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 01 16:27:27.486987 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application result = self._query(query) Jul 01 16:27:27.486987 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 01 16:27:27.486987 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application conn.query(q) Jul 01 16:27:27.486987 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 01 16:27:27.486987 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 01 16:27:27.486987 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 01 16:27:27.486987 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application result.read() Jul 01 16:27:27.486987 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 01 16:27:27.486987 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 01 16:27:27.486987 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 01 16:27:27.486987 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 01 16:27:27.489012 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 01 16:27:27.489012 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 01 16:27:27.489012 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 01 16:27:27.489012 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 01 16:27:27.489012 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 01 16:27:27.489012 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jul 01 16:27:27.489012 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application [parameters: {'id': '0b0b9337d5aa4f8386deabe7c3390e2f', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jul 01 16:27:27.489012 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 01 16:27:27.489012 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application Jul 01 16:27:27.489012 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 01 16:27:27.489012 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application Jul 01 16:27:27.489012 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 16:27:27.489012 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 16:27:27.489012 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 16:27:27.489012 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 16:27:27.489012 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 16:27:27.491025 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 16:27:27.491025 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 16:27:27.491025 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 16:27:27.491025 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 16:27:27.491025 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 16:27:27.491025 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 16:27:27.491025 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Jul 01 16:27:27.491025 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Jul 01 16:27:27.491025 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 16:27:27.491025 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 16:27:27.491025 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Jul 01 16:27:27.491025 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Jul 01 16:27:27.491025 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 01 16:27:27.491025 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 01 16:27:27.491025 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jul 01 16:27:27.491025 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application Jul 01 16:27:27.497570 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 243/478] 10.222.0.57 () {60 vars in 1131 bytes} [Fri Jul 1 16:27:27 2022] POST /identity/v3/roles => generated 144 bytes in 247 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Jul 01 16:27:27.503371 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292b8612-2c06-4639-924b-8b057a9c2470 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:27:27.639458 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292b8612-2c06-4639-924b-8b057a9c2470 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:27:27.671001 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:27:27.681394 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e4c03b-297e-4caf-ae03-9c2696f390c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:27:27.682217 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e4c03b-297e-4caf-ae03-9c2696f390c7 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:27.682535 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e4c03b-297e-4caf-ae03-9c2696f390c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:27.682825 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e4c03b-297e-4caf-ae03-9c2696f390c7 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:27.695092 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e4c03b-297e-4caf-ae03-9c2696f390c7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 01 16:27:27.704207 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application [None req-75e4c03b-297e-4caf-ae03-9c2696f390c7 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 01 16:27:27.704207 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 16:27:27.704207 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 16:27:27.704207 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 16:27:27.704207 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 16:27:27.704207 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 16:27:27.704207 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 16:27:27.704207 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 16:27:27.704207 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 16:27:27.704207 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 16:27:27.704207 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 16:27:27.704207 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 16:27:27.704207 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 01 16:27:27.704207 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 01 16:27:27.704207 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 01 16:27:27.704207 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 01 16:27:27.705352 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 01 16:27:27.705352 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 01 16:27:27.705352 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 01 16:27:27.705352 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 01 16:27:27.705352 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 16:27:27.705352 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 16:27:27.705352 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 16:27:27.705352 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 16:27:27.705352 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 16:27:27.705352 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 16:27:27.705352 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 16:27:27.705352 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application with Lock( Jul 01 16:27:27.705352 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 16:27:27.705352 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._enter() Jul 01 16:27:27.705352 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 16:27:27.705352 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 16:27:27.706165 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 16:27:27.706165 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.creator() Jul 01 16:27:27.706165 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 16:27:27.706165 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application created_value = creator( Jul 01 16:27:27.706165 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 01 16:27:27.706165 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 01 16:27:27.706165 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 01 16:27:27.706165 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 01 16:27:27.706165 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 01 16:27:27.706165 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 01 16:27:27.706165 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 01 16:27:27.706165 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Jul 01 16:27:27.710156 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 236/479] 10.222.0.57 () {56 vars in 1114 bytes} [Fri Jul 1 16:27:27 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 16:27:27.711917 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a84f9a-e86c-4065-8e31-2387614a88f5 None admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:27:27.802253 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:27:27.829309 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a84f9a-e86c-4065-8e31-2387614a88f5 None admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:27:27.876913 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e88674-1f47-4f0a-b424-b94692abd6c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:27:27.877597 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e88674-1f47-4f0a-b424-b94692abd6c3 None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:27.877884 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e88674-1f47-4f0a-b424-b94692abd6c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:27.878164 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e88674-1f47-4f0a-b424-b94692abd6c3 None admin] PATH_INFO: `/v3/roles` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:27.878962 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e88674-1f47-4f0a-b424-b94692abd6c3 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80312) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:27:27.879272 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e88674-1f47-4f0a-b424-b94692abd6c3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:27:27.880650 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e88674-1f47-4f0a-b424-b94692abd6c3 None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:27:27.891144 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 244/480] 10.222.0.57 () {56 vars in 1111 bytes} [Fri Jul 1 16:27:27 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:27:28.738401 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbdf190-c4ab-4531-9082-1479de02a9de None None] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:28.738401 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbdf190-c4ab-4531-9082-1479de02a9de None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:28.738401 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbdf190-c4ab-4531-9082-1479de02a9de None None] PATH_INFO: `/` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:28.739362 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 237/481] 10.222.0.57 () {54 vars in 926 bytes} [Fri Jul 1 16:27:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 16:27:28.745798 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca93a05b-16ae-4250-96a8-6652319d1ced None None] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:28.746581 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca93a05b-16ae-4250-96a8-6652319d1ced None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:28.747026 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca93a05b-16ae-4250-96a8-6652319d1ced None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:28.765268 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.auth.core [None req-ca93a05b-16ae-4250-96a8-6652319d1ced None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80312) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:27:28.766717 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-ca93a05b-16ae-4250-96a8-6652319d1ced None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:27:28.846453 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 245/482] 10.222.0.57 () {58 vars in 1023 bytes} [Fri Jul 1 16:27:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 01 16:27:28.867059 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dbdf190-c4ab-4531-9082-1479de02a9de None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:27:28.989435 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dbdf190-c4ab-4531-9082-1479de02a9de None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:27:29.025660 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e1b9d2-b8c5-4018-a07d-527f4e070031 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:27:29.026716 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e1b9d2-b8c5-4018-a07d-527f4e070031 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:29.027144 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e1b9d2-b8c5-4018-a07d-527f4e070031 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:29.027568 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e1b9d2-b8c5-4018-a07d-527f4e070031 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:29.038423 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e1b9d2-b8c5-4018-a07d-527f4e070031 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 01 16:27:29.051115 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application [None req-72e1b9d2-b8c5-4018-a07d-527f4e070031 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 01 16:27:29.051115 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 16:27:29.051115 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 16:27:29.051115 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 16:27:29.051115 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 16:27:29.051115 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 16:27:29.051115 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 16:27:29.051115 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 16:27:29.051115 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 16:27:29.051115 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 16:27:29.051115 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 16:27:29.051115 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 16:27:29.051115 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 01 16:27:29.051115 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 01 16:27:29.051115 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 01 16:27:29.051115 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 01 16:27:29.052280 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 01 16:27:29.052280 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 01 16:27:29.052280 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 01 16:27:29.052280 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 01 16:27:29.052280 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 16:27:29.052280 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 16:27:29.052280 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 16:27:29.052280 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 16:27:29.052280 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 16:27:29.052280 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 16:27:29.052280 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 16:27:29.052280 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application with Lock( Jul 01 16:27:29.052280 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 16:27:29.052280 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._enter() Jul 01 16:27:29.052280 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 16:27:29.052280 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 16:27:29.053277 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 16:27:29.053277 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.creator() Jul 01 16:27:29.053277 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 16:27:29.053277 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application created_value = creator( Jul 01 16:27:29.053277 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 01 16:27:29.053277 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 01 16:27:29.053277 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 01 16:27:29.053277 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 01 16:27:29.053277 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 01 16:27:29.053277 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 01 16:27:29.053277 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 01 16:27:29.053277 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Jul 01 16:27:29.055462 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 238/483] 10.222.0.57 () {56 vars in 1102 bytes} [Fri Jul 1 16:27:28 2022] GET /identity/v3/roles/service => generated 85 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 16:27:29.061484 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca93a05b-16ae-4250-96a8-6652319d1ced None None] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:27:29.170854 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=100 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:27:29.218236 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca93a05b-16ae-4250-96a8-6652319d1ced None None] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:27:29.271563 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775fe6dd-1763-4760-8771-a77c0a7529f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:27:29.272170 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775fe6dd-1763-4760-8771-a77c0a7529f3 None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:29.272459 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775fe6dd-1763-4760-8771-a77c0a7529f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:29.272741 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775fe6dd-1763-4760-8771-a77c0a7529f3 None admin] PATH_INFO: `/v3/roles` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:29.273549 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-775fe6dd-1763-4760-8771-a77c0a7529f3 None admin] RBAC: Adding query filter params (name=service) {{(pid=80312) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:27:29.273862 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-775fe6dd-1763-4760-8771-a77c0a7529f3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:27:29.275298 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-775fe6dd-1763-4760-8771-a77c0a7529f3 None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:27:29.289940 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 246/484] 10.222.0.57 () {56 vars in 1103 bytes} [Fri Jul 1 16:27:29 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:27:29.296117 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e1b9d2-b8c5-4018-a07d-527f4e070031 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:27:29.341896 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcbdfe8-cafc-415b-af9b-49b3104a0069 None None] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:29.342209 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcbdfe8-cafc-415b-af9b-49b3104a0069 None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:29.342500 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcbdfe8-cafc-415b-af9b-49b3104a0069 None None] PATH_INFO: `/` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:29.344050 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 247/485] 10.222.0.57 () {54 vars in 926 bytes} [Fri Jul 1 16:27:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 16:27:29.349702 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f930c7f5-749e-42b0-b55b-631af6eb7870 None None] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:29.349997 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f930c7f5-749e-42b0-b55b-631af6eb7870 None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:29.350288 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f930c7f5-749e-42b0-b55b-631af6eb7870 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:29.354377 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:27:29.387361 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.auth.core [None req-f930c7f5-749e-42b0-b55b-631af6eb7870 None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80312) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:27:29.388569 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-f930c7f5-749e-42b0-b55b-631af6eb7870 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:27:29.421025 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e1b9d2-b8c5-4018-a07d-527f4e070031 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:27:29.457908 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c2255c-d7c4-4c4b-a8dc-5c7fb42cf3d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:27:29.459032 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c2255c-d7c4-4c4b-a8dc-5c7fb42cf3d9 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:29.459535 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c2255c-d7c4-4c4b-a8dc-5c7fb42cf3d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:29.459946 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c2255c-d7c4-4c4b-a8dc-5c7fb42cf3d9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:29.472669 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c2255c-d7c4-4c4b-a8dc-5c7fb42cf3d9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:27:29.474392 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c2255c-d7c4-4c4b-a8dc-5c7fb42cf3d9 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:27:29.503756 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application [None req-06c2255c-d7c4-4c4b-a8dc-5c7fb42cf3d9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 16:27:29.503756 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 16:27:29.503756 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 16:27:29.503756 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 16:27:29.503756 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 16:27:29.503756 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 16:27:29.503756 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 16:27:29.503756 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 16:27:29.503756 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 16:27:29.503756 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 16:27:29.503756 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 16:27:29.503756 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 16:27:29.503756 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 01 16:27:29.503756 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 01 16:27:29.503756 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 01 16:27:29.504828 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 01 16:27:29.504828 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 16:27:29.504828 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 16:27:29.504828 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 16:27:29.504828 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 16:27:29.504828 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 16:27:29.504828 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 16:27:29.504828 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 16:27:29.504828 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application with Lock( Jul 01 16:27:29.504828 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 16:27:29.504828 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._enter() Jul 01 16:27:29.504828 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 16:27:29.504828 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 16:27:29.504828 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 16:27:29.504828 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.creator() Jul 01 16:27:29.504828 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 16:27:29.504828 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application created_value = creator( Jul 01 16:27:29.505753 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 01 16:27:29.505753 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 01 16:27:29.505753 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 16:27:29.505753 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Jul 01 16:27:29.509580 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 239/486] 10.222.0.57 () {56 vars in 1108 bytes} [Fri Jul 1 16:27:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 16:27:29.511998 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c2255c-d7c4-4c4b-a8dc-5c7fb42cf3d9 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:27:29.537054 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 248/487] 10.222.0.57 () {58 vars in 1023 bytes} [Fri Jul 1 16:27:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 01 16:27:29.577713 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f930c7f5-749e-42b0-b55b-631af6eb7870 None None] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:27:29.682718 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:27:29.694913 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=100 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:27:29.699498 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c2255c-d7c4-4c4b-a8dc-5c7fb42cf3d9 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:27:29.745713 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec6165b-059a-41f2-a3be-2c2e8508175b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:27:29.746535 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec6165b-059a-41f2-a3be-2c2e8508175b None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:29.746711 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec6165b-059a-41f2-a3be-2c2e8508175b None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:29.746992 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec6165b-059a-41f2-a3be-2c2e8508175b None admin] PATH_INFO: `/v3/domains` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:29.747729 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec6165b-059a-41f2-a3be-2c2e8508175b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:27:29.748035 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec6165b-059a-41f2-a3be-2c2e8508175b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:27:29.763701 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec6165b-059a-41f2-a3be-2c2e8508175b None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:27:29.768833 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f930c7f5-749e-42b0-b55b-631af6eb7870 None None] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:27:29.792725 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 240/488] 10.222.0.57 () {56 vars in 1109 bytes} [Fri Jul 1 16:27:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:27:29.807555 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec6165b-059a-41f2-a3be-2c2e8508175b None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:27:29.822702 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2094aa-e047-442c-ba45-0317d829b18d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:27:29.823525 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2094aa-e047-442c-ba45-0317d829b18d None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:29.823812 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2094aa-e047-442c-ba45-0317d829b18d None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:29.824091 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2094aa-e047-442c-ba45-0317d829b18d None admin] PATH_INFO: `/v3/roles/service` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:29.845984 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce2094aa-e047-442c-ba45-0317d829b18d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 01 16:27:29.856267 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application [None req-ce2094aa-e047-442c-ba45-0317d829b18d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 01 16:27:29.856267 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 16:27:29.856267 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 16:27:29.856267 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 16:27:29.856267 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 16:27:29.856267 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 16:27:29.856267 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 16:27:29.856267 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 16:27:29.856267 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 16:27:29.856267 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 16:27:29.856267 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 16:27:29.856267 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 16:27:29.856267 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 01 16:27:29.856267 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 01 16:27:29.856267 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 01 16:27:29.856267 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 01 16:27:29.857209 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 01 16:27:29.857209 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 01 16:27:29.857209 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 01 16:27:29.857209 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 01 16:27:29.857209 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 16:27:29.857209 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 16:27:29.857209 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 16:27:29.857209 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 16:27:29.857209 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 16:27:29.857209 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 16:27:29.857209 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 16:27:29.857209 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application with Lock( Jul 01 16:27:29.857209 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 16:27:29.857209 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self._enter() Jul 01 16:27:29.857209 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 16:27:29.857209 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 16:27:29.857999 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 16:27:29.857999 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.creator() Jul 01 16:27:29.857999 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 16:27:29.857999 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application created_value = creator( Jul 01 16:27:29.857999 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 01 16:27:29.857999 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 01 16:27:29.857999 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 01 16:27:29.857999 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 01 16:27:29.857999 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 01 16:27:29.857999 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 01 16:27:29.857999 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 01 16:27:29.857999 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application Jul 01 16:27:29.857999 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 249/489] 10.222.0.57 () {56 vars in 1102 bytes} [Fri Jul 1 16:27:29 2022] GET /identity/v3/roles/service => generated 85 bytes in 280 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 01 16:27:29.859805 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afdaf2c-d715-49f4-99aa-277dc8cc80da None None] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:29.860093 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afdaf2c-d715-49f4-99aa-277dc8cc80da None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:29.860485 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afdaf2c-d715-49f4-99aa-277dc8cc80da None None] PATH_INFO: `/` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:29.861886 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 250/490] 10.222.0.57 () {54 vars in 926 bytes} [Fri Jul 1 16:27:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 16:27:29.867029 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1afdaf2c-d715-49f4-99aa-277dc8cc80da None None] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:27:29.943404 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec6165b-059a-41f2-a3be-2c2e8508175b None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:27:29.982921 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1afdaf2c-d715-49f4-99aa-277dc8cc80da None None] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:27:29.985187 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640cd58d-4844-418f-b919-b9b2b02d8384 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:27:29.986268 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640cd58d-4844-418f-b919-b9b2b02d8384 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:29.986767 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640cd58d-4844-418f-b919-b9b2b02d8384 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:29.987466 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640cd58d-4844-418f-b919-b9b2b02d8384 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:29.996680 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-640cd58d-4844-418f-b919-b9b2b02d8384 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:27:29.998569 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-640cd58d-4844-418f-b919-b9b2b02d8384 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:27:30.009322 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=100 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:27:30.016395 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application [None req-640cd58d-4844-418f-b919-b9b2b02d8384 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 01 16:27:30.016395 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 16:27:30.016395 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 16:27:30.016395 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 16:27:30.016395 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 16:27:30.016395 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 16:27:30.016395 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 16:27:30.016395 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 16:27:30.016395 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 16:27:30.016395 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 16:27:30.016395 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 16:27:30.016395 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 16:27:30.016395 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 01 16:27:30.016395 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 01 16:27:30.016395 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 01 16:27:30.016395 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 01 16:27:30.017321 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 16:27:30.017321 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 16:27:30.017321 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 01 16:27:30.017321 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 01 16:27:30.017321 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 01 16:27:30.017321 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 01 16:27:30.017321 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 16:27:30.017321 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 16:27:30.017321 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 16:27:30.017321 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 16:27:30.017321 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 16:27:30.017321 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application with Lock( Jul 01 16:27:30.017321 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 16:27:30.017321 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._enter() Jul 01 16:27:30.017321 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 16:27:30.017321 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 16:27:30.017321 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 16:27:30.017321 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.creator() Jul 01 16:27:30.018226 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 16:27:30.018226 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application created_value = creator( Jul 01 16:27:30.018226 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 01 16:27:30.018226 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 01 16:27:30.018226 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 01 16:27:30.018226 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 01 16:27:30.018226 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 01 16:27:30.018226 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 01 16:27:30.018226 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 01 16:27:30.018226 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Jul 01 16:27:30.020693 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 241/491] 10.222.0.57 () {56 vars in 1093 bytes} [Fri Jul 1 16:27:29 2022] GET /identity/v3/users/nova => generated 82 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 16:27:30.022164 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5547f8-960c-4955-a0d5-97f4a33e0988 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:30.022694 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5547f8-960c-4955-a0d5-97f4a33e0988 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:30.023679 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5547f8-960c-4955-a0d5-97f4a33e0988 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:30.032291 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15b15f0-eea7-4c08-946a-912e78bca9a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:27:30.032970 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15b15f0-eea7-4c08-946a-912e78bca9a6 None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:30.033258 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15b15f0-eea7-4c08-946a-912e78bca9a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:30.033585 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15b15f0-eea7-4c08-946a-912e78bca9a6 None admin] PATH_INFO: `/v3/roles` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:30.034384 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b15b15f0-eea7-4c08-946a-912e78bca9a6 None admin] RBAC: Adding query filter params (name=service) {{(pid=80312) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:27:30.034789 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b15b15f0-eea7-4c08-946a-912e78bca9a6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:27:30.036217 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b15b15f0-eea7-4c08-946a-912e78bca9a6 None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:27:30.047098 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 251/492] 10.222.0.57 () {56 vars in 1103 bytes} [Fri Jul 1 16:27:29 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:27:30.047752 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-ab5547f8-960c-4955-a0d5-97f4a33e0988 None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:27:30.047988 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15b15f0-eea7-4c08-946a-912e78bca9a6 None admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:27:30.049404 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-ab5547f8-960c-4955-a0d5-97f4a33e0988 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:27:30.058456 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:27:30.130461 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 242/493] 10.222.0.57 () {58 vars in 1023 bytes} [Fri Jul 1 16:27:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 01 16:27:30.131724 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5547f8-960c-4955-a0d5-97f4a33e0988 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:27:30.166400 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15b15f0-eea7-4c08-946a-912e78bca9a6 None admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:27:30.194990 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fdacfeb-ec0f-4612-b2e3-a7705ac3dbf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:27:30.195696 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdacfeb-ec0f-4612-b2e3-a7705ac3dbf4 None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:30.196049 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdacfeb-ec0f-4612-b2e3-a7705ac3dbf4 None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:30.196424 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdacfeb-ec0f-4612-b2e3-a7705ac3dbf4 None admin] PATH_INFO: `/v3/users` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:30.197424 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fdacfeb-ec0f-4612-b2e3-a7705ac3dbf4 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80312) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:27:30.197627 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fdacfeb-ec0f-4612-b2e3-a7705ac3dbf4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:27:30.199085 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fdacfeb-ec0f-4612-b2e3-a7705ac3dbf4 None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:27:30.243423 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 252/494] 10.222.0.57 () {56 vars in 1133 bytes} [Fri Jul 1 16:27:30 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:27:30.244273 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fdacfeb-ec0f-4612-b2e3-a7705ac3dbf4 None admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:27:30.266149 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5547f8-960c-4955-a0d5-97f4a33e0988 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:27:30.289411 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=100 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:27:30.305146 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c993de52-454d-4a54-888c-5b359a8012eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:27:30.306777 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c993de52-454d-4a54-888c-5b359a8012eb None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:30.307291 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c993de52-454d-4a54-888c-5b359a8012eb None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:30.307782 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c993de52-454d-4a54-888c-5b359a8012eb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:30.319336 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c993de52-454d-4a54-888c-5b359a8012eb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:27:30.321257 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c993de52-454d-4a54-888c-5b359a8012eb None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:27:30.323148 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:27:30.340903 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application [None req-c993de52-454d-4a54-888c-5b359a8012eb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 16:27:30.340903 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 16:27:30.340903 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 16:27:30.340903 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 16:27:30.340903 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 16:27:30.340903 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 16:27:30.340903 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 16:27:30.340903 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 16:27:30.340903 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 16:27:30.340903 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 16:27:30.340903 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 16:27:30.340903 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 16:27:30.340903 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 01 16:27:30.340903 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 01 16:27:30.340903 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 01 16:27:30.341818 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 01 16:27:30.341818 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 16:27:30.341818 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 16:27:30.341818 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 16:27:30.341818 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 16:27:30.341818 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 16:27:30.341818 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 16:27:30.341818 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 16:27:30.341818 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application with Lock( Jul 01 16:27:30.341818 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 16:27:30.341818 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._enter() Jul 01 16:27:30.341818 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 16:27:30.341818 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 16:27:30.341818 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 16:27:30.341818 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.creator() Jul 01 16:27:30.341818 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 16:27:30.341818 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application created_value = creator( Jul 01 16:27:30.343083 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 01 16:27:30.343083 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 01 16:27:30.343083 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 16:27:30.343083 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Jul 01 16:27:30.343083 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 243/495] 10.222.0.57 () {56 vars in 1108 bytes} [Fri Jul 1 16:27:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 16:27:30.343083 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c993de52-454d-4a54-888c-5b359a8012eb None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:27:30.362325 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fdacfeb-ec0f-4612-b2e3-a7705ac3dbf4 None admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:27:30.403670 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65555342-2b2a-402b-a812-c68128058da4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:27:30.404477 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65555342-2b2a-402b-a812-c68128058da4 None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:30.404761 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65555342-2b2a-402b-a812-c68128058da4 None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:30.405042 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65555342-2b2a-402b-a812-c68128058da4 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:30.417062 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65555342-2b2a-402b-a812-c68128058da4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 01 16:27:30.425153 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application [None req-65555342-2b2a-402b-a812-c68128058da4 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 01 16:27:30.425153 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 16:27:30.425153 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 16:27:30.425153 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 16:27:30.425153 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 16:27:30.425153 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 16:27:30.425153 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 16:27:30.425153 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 16:27:30.425153 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 16:27:30.425153 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 16:27:30.425153 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 16:27:30.425153 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 16:27:30.425153 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 01 16:27:30.425153 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 01 16:27:30.425153 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 01 16:27:30.425153 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 01 16:27:30.426038 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 01 16:27:30.426038 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 01 16:27:30.426038 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 01 16:27:30.426038 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 01 16:27:30.426038 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 16:27:30.426038 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 16:27:30.426038 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 16:27:30.426038 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 16:27:30.426038 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 16:27:30.426038 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 16:27:30.426038 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 16:27:30.426038 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application with Lock( Jul 01 16:27:30.426038 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 16:27:30.426038 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self._enter() Jul 01 16:27:30.426038 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 16:27:30.426038 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 16:27:30.431199 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 16:27:30.431199 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.creator() Jul 01 16:27:30.431199 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 16:27:30.431199 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application created_value = creator( Jul 01 16:27:30.431199 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 01 16:27:30.431199 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 01 16:27:30.431199 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 01 16:27:30.431199 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 01 16:27:30.431199 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 01 16:27:30.431199 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 01 16:27:30.431199 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 01 16:27:30.431199 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application Jul 01 16:27:30.431199 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 253/496] 10.222.0.57 () {56 vars in 1102 bytes} [Fri Jul 1 16:27:30 2022] GET /identity/v3/roles/service => generated 85 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 16:27:30.431199 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f50e554-6923-4908-a9e4-5d7bc3ab36d3 None None] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:30.431950 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f50e554-6923-4908-a9e4-5d7bc3ab36d3 None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:30.431950 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f50e554-6923-4908-a9e4-5d7bc3ab36d3 None None] PATH_INFO: `/` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:30.431950 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 254/497] 10.222.0.57 () {54 vars in 926 bytes} [Fri Jul 1 16:27:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 16:27:30.431950 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f50e554-6923-4908-a9e4-5d7bc3ab36d3 None None] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:27:30.460199 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c993de52-454d-4a54-888c-5b359a8012eb None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:27:30.502985 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67ab01c-e402-48e9-8b85-fc2ae177a778 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:27:30.503645 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67ab01c-e402-48e9-8b85-fc2ae177a778 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:30.503930 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67ab01c-e402-48e9-8b85-fc2ae177a778 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:30.504214 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67ab01c-e402-48e9-8b85-fc2ae177a778 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:30.559407 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=100 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:27:30.564046 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f50e554-6923-4908-a9e4-5d7bc3ab36d3 None None] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:27:30.574293 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:27:30.583699 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f67ab01c-e402-48e9-8b85-fc2ae177a778 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:27:30.585174 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f67ab01c-e402-48e9-8b85-fc2ae177a778 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:27:30.604312 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9733a48-5daf-4cc3-8d80-54edb6e11a4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:27:30.605065 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9733a48-5daf-4cc3-8d80-54edb6e11a4a None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:30.605273 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9733a48-5daf-4cc3-8d80-54edb6e11a4a None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:30.605549 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9733a48-5daf-4cc3-8d80-54edb6e11a4a None admin] PATH_INFO: `/v3/domains` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:30.606281 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9733a48-5daf-4cc3-8d80-54edb6e11a4a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80312) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:27:30.606693 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9733a48-5daf-4cc3-8d80-54edb6e11a4a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:27:30.608019 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9733a48-5daf-4cc3-8d80-54edb6e11a4a None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:27:30.620987 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 255/498] 10.222.0.57 () {56 vars in 1109 bytes} [Fri Jul 1 16:27:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:27:30.623518 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9733a48-5daf-4cc3-8d80-54edb6e11a4a None admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:27:30.708999 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 244/499] 10.222.0.57 () {58 vars in 1282 bytes} [Fri Jul 1 16:27:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 01 16:27:30.713653 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca437942-6d43-4331-901c-b7fed24351bf None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:30.714187 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca437942-6d43-4331-901c-b7fed24351bf None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:30.714735 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca437942-6d43-4331-901c-b7fed24351bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:30.733576 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-ca437942-6d43-4331-901c-b7fed24351bf None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:27:30.734815 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-ca437942-6d43-4331-901c-b7fed24351bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:27:30.746239 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9733a48-5daf-4cc3-8d80-54edb6e11a4a None admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:27:30.795178 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880ab0d3-fe54-4506-9ab8-50e7e016e6d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:27:30.795969 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880ab0d3-fe54-4506-9ab8-50e7e016e6d2 None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:30.796332 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880ab0d3-fe54-4506-9ab8-50e7e016e6d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:30.796687 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880ab0d3-fe54-4506-9ab8-50e7e016e6d2 None admin] PATH_INFO: `/v3/roles` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:30.797596 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-880ab0d3-fe54-4506-9ab8-50e7e016e6d2 None admin] RBAC: Adding query filter params (name=service) {{(pid=80312) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:27:30.798198 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-880ab0d3-fe54-4506-9ab8-50e7e016e6d2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:27:30.800822 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-880ab0d3-fe54-4506-9ab8-50e7e016e6d2 None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:27:30.818989 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 256/500] 10.222.0.57 () {56 vars in 1103 bytes} [Fri Jul 1 16:27:30 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:27:30.821954 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880ab0d3-fe54-4506-9ab8-50e7e016e6d2 None admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:27:30.860909 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 245/501] 10.222.0.57 () {58 vars in 1023 bytes} [Fri Jul 1 16:27:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 01 16:27:30.861816 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca437942-6d43-4331-901c-b7fed24351bf None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:27:30.862718 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=100 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:27:30.899288 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:27:30.957281 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880ab0d3-fe54-4506-9ab8-50e7e016e6d2 None admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:27:30.989703 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca437942-6d43-4331-901c-b7fed24351bf None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:27:30.998445 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb72f0a-f1b6-48a5-8a3a-751764b8a409 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:27:30.999245 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb72f0a-f1b6-48a5-8a3a-751764b8a409 None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:30.999529 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb72f0a-f1b6-48a5-8a3a-751764b8a409 None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:30.999806 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb72f0a-f1b6-48a5-8a3a-751764b8a409 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:31.008731 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb72f0a-f1b6-48a5-8a3a-751764b8a409 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:27:31.010323 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb72f0a-f1b6-48a5-8a3a-751764b8a409 None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:27:31.026182 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application [None req-7eb72f0a-f1b6-48a5-8a3a-751764b8a409 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 01 16:27:31.026182 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 16:27:31.026182 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 16:27:31.026182 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 16:27:31.026182 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 16:27:31.026182 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 16:27:31.026182 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 16:27:31.026182 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 16:27:31.026182 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 16:27:31.026182 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 16:27:31.026182 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 16:27:31.026182 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 16:27:31.026182 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 01 16:27:31.026182 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 01 16:27:31.026182 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 01 16:27:31.026182 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 01 16:27:31.027284 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 16:27:31.027284 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 16:27:31.027284 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 01 16:27:31.027284 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 01 16:27:31.027284 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 01 16:27:31.027284 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 01 16:27:31.027284 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 16:27:31.027284 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 16:27:31.027284 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 16:27:31.027284 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 16:27:31.027284 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 16:27:31.027284 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application with Lock( Jul 01 16:27:31.027284 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 16:27:31.027284 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self._enter() Jul 01 16:27:31.027284 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 16:27:31.027284 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 16:27:31.027284 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 16:27:31.027284 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.creator() Jul 01 16:27:31.028388 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 16:27:31.028388 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application created_value = creator( Jul 01 16:27:31.028388 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 01 16:27:31.028388 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 01 16:27:31.028388 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 01 16:27:31.028388 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 01 16:27:31.028388 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 01 16:27:31.028388 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 01 16:27:31.028388 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 01 16:27:31.028388 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application Jul 01 16:27:31.029478 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf28fca-db43-4a34-a2e6-497cb5072d1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:27:31.029478 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf28fca-db43-4a34-a2e6-497cb5072d1c None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:31.029478 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf28fca-db43-4a34-a2e6-497cb5072d1c None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:31.029691 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 257/502] 10.222.0.57 () {56 vars in 1099 bytes} [Fri Jul 1 16:27:30 2022] GET /identity/v3/users/glance => generated 84 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 16:27:31.029756 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf28fca-db43-4a34-a2e6-497cb5072d1c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:31.029829 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb72f0a-f1b6-48a5-8a3a-751764b8a409 None admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:27:31.040782 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf28fca-db43-4a34-a2e6-497cb5072d1c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:27:31.042238 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf28fca-db43-4a34-a2e6-497cb5072d1c None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:27:31.061443 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application [None req-edf28fca-db43-4a34-a2e6-497cb5072d1c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 16:27:31.061443 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 16:27:31.061443 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 16:27:31.061443 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 16:27:31.061443 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 16:27:31.061443 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 16:27:31.061443 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 16:27:31.061443 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 16:27:31.061443 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 16:27:31.061443 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 16:27:31.061443 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 16:27:31.061443 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 16:27:31.061443 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 01 16:27:31.061443 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 01 16:27:31.061443 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 01 16:27:31.062480 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 01 16:27:31.062480 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 16:27:31.062480 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 16:27:31.062480 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 16:27:31.062480 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 16:27:31.062480 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 16:27:31.062480 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 16:27:31.062480 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 16:27:31.062480 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application with Lock( Jul 01 16:27:31.062480 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 16:27:31.062480 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._enter() Jul 01 16:27:31.062480 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 16:27:31.062480 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 16:27:31.062480 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 16:27:31.062480 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.creator() Jul 01 16:27:31.062480 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 16:27:31.062480 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application created_value = creator( Jul 01 16:27:31.063319 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 01 16:27:31.063319 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 01 16:27:31.063319 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 16:27:31.063319 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Jul 01 16:27:31.063319 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 246/503] 10.222.0.57 () {56 vars in 1108 bytes} [Fri Jul 1 16:27:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 16:27:31.063589 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf28fca-db43-4a34-a2e6-497cb5072d1c None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:27:31.133053 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=100 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:27:31.146373 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:27:31.152501 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb72f0a-f1b6-48a5-8a3a-751764b8a409 None admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:27:31.172133 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf28fca-db43-4a34-a2e6-497cb5072d1c None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:27:31.193151 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d1d2dd-0389-46b2-af29-d7a906f25ea7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:27:31.193151 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d1d2dd-0389-46b2-af29-d7a906f25ea7 None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:31.193448 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d1d2dd-0389-46b2-af29-d7a906f25ea7 None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:31.193630 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d1d2dd-0389-46b2-af29-d7a906f25ea7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:31.206033 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d1d2dd-0389-46b2-af29-d7a906f25ea7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:27:31.207561 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d1d2dd-0389-46b2-af29-d7a906f25ea7 None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:27:31.216277 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cebdadb-06f1-41ef-b9c0-5eb1aff03230 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:27:31.217355 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cebdadb-06f1-41ef-b9c0-5eb1aff03230 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:31.217807 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cebdadb-06f1-41ef-b9c0-5eb1aff03230 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:31.218379 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cebdadb-06f1-41ef-b9c0-5eb1aff03230 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:31.228185 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application [None req-b9d1d2dd-0389-46b2-af29-d7a906f25ea7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 16:27:31.228185 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 16:27:31.228185 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 16:27:31.228185 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 16:27:31.228185 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 16:27:31.228185 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 16:27:31.228185 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 16:27:31.228185 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 16:27:31.228185 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 16:27:31.228185 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 16:27:31.228185 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 16:27:31.228185 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 16:27:31.228185 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 01 16:27:31.228185 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 01 16:27:31.228185 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 01 16:27:31.229074 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 01 16:27:31.229074 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 16:27:31.229074 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 16:27:31.229074 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 16:27:31.229074 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 16:27:31.229074 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 16:27:31.229074 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 16:27:31.229074 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 16:27:31.229074 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application with Lock( Jul 01 16:27:31.229074 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 16:27:31.229074 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self._enter() Jul 01 16:27:31.229074 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 16:27:31.229074 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 16:27:31.229074 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 16:27:31.229074 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.creator() Jul 01 16:27:31.229074 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 16:27:31.229074 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application created_value = creator( Jul 01 16:27:31.229935 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 01 16:27:31.229935 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 01 16:27:31.229935 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 16:27:31.229935 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application Jul 01 16:27:31.229935 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 258/504] 10.222.0.57 () {56 vars in 1108 bytes} [Fri Jul 1 16:27:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 16:27:31.231366 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cebdadb-06f1-41ef-b9c0-5eb1aff03230 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 01 16:27:31.232202 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15892ba3-e0a8-4d16-903f-ee4dbe87b53a None None] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:31.232563 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15892ba3-e0a8-4d16-903f-ee4dbe87b53a None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:31.232851 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15892ba3-e0a8-4d16-903f-ee4dbe87b53a None None] PATH_INFO: `/` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:31.238100 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 259/505] 10.222.0.57 () {54 vars in 926 bytes} [Fri Jul 1 16:27:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 16:27:31.238100 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15892ba3-e0a8-4d16-903f-ee4dbe87b53a None None] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:27:31.239885 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application [None req-3cebdadb-06f1-41ef-b9c0-5eb1aff03230 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 01 16:27:31.239885 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 16:27:31.239885 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 16:27:31.239885 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 16:27:31.239885 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 16:27:31.239885 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 16:27:31.239885 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 16:27:31.239885 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 16:27:31.239885 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 16:27:31.239885 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 16:27:31.239885 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 16:27:31.239885 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 16:27:31.239885 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 01 16:27:31.239885 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 01 16:27:31.239885 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 01 16:27:31.239885 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 01 16:27:31.240787 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 01 16:27:31.240787 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 01 16:27:31.240787 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 01 16:27:31.240787 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 01 16:27:31.240787 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 16:27:31.240787 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 16:27:31.240787 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 16:27:31.240787 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 16:27:31.240787 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 16:27:31.240787 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 16:27:31.240787 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 16:27:31.240787 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application with Lock( Jul 01 16:27:31.240787 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 16:27:31.240787 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._enter() Jul 01 16:27:31.240787 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 16:27:31.240787 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 16:27:31.241598 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 16:27:31.241598 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.creator() Jul 01 16:27:31.241598 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 16:27:31.241598 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application created_value = creator( Jul 01 16:27:31.241598 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 01 16:27:31.241598 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 01 16:27:31.241598 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 01 16:27:31.241598 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 01 16:27:31.241598 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 01 16:27:31.241598 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 01 16:27:31.241598 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 01 16:27:31.241598 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Jul 01 16:27:31.242851 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 247/506] 10.222.0.57 () {56 vars in 1102 bytes} [Fri Jul 1 16:27:31 2022] GET /identity/v3/roles/service => generated 85 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 16:27:31.244080 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cebdadb-06f1-41ef-b9c0-5eb1aff03230 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:27:31.352279 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cebdadb-06f1-41ef-b9c0-5eb1aff03230 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:27:31.353679 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15892ba3-e0a8-4d16-903f-ee4dbe87b53a None None] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:27:31.385465 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73891b9d-90fb-48d3-ad4e-819ebc5c51e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:27:31.386042 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73891b9d-90fb-48d3-ad4e-819ebc5c51e0 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:31.386325 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73891b9d-90fb-48d3-ad4e-819ebc5c51e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:31.386626 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73891b9d-90fb-48d3-ad4e-819ebc5c51e0 None admin] PATH_INFO: `/v3/domains` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:31.387341 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73891b9d-90fb-48d3-ad4e-819ebc5c51e0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:27:31.387644 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73891b9d-90fb-48d3-ad4e-819ebc5c51e0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:27:31.387848 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c72bc1e-a7c2-4221-be38-efdf7285b2e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:27:31.388522 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c72bc1e-a7c2-4221-be38-efdf7285b2e5 None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:31.388812 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c72bc1e-a7c2-4221-be38-efdf7285b2e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:31.389016 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73891b9d-90fb-48d3-ad4e-819ebc5c51e0 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:27:31.389102 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c72bc1e-a7c2-4221-be38-efdf7285b2e5 None admin] PATH_INFO: `/v3/users` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:31.389921 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c72bc1e-a7c2-4221-be38-efdf7285b2e5 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80312) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:27:31.390228 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c72bc1e-a7c2-4221-be38-efdf7285b2e5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:27:31.391415 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:27:31.391701 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c72bc1e-a7c2-4221-be38-efdf7285b2e5 None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:27:31.393514 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=100 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:27:31.402999 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 248/507] 10.222.0.57 () {56 vars in 1109 bytes} [Fri Jul 1 16:27:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:27:31.403823 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73891b9d-90fb-48d3-ad4e-819ebc5c51e0 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:27:31.411590 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 260/508] 10.222.0.57 () {56 vars in 1137 bytes} [Fri Jul 1 16:27:31 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:27:31.412859 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb1b6c8-2d6b-47b9-8a21-b33e8ecc3a49 None None] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:31.413153 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb1b6c8-2d6b-47b9-8a21-b33e8ecc3a49 None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:31.413432 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb1b6c8-2d6b-47b9-8a21-b33e8ecc3a49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:31.430114 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.auth.core [None req-6eb1b6c8-2d6b-47b9-8a21-b33e8ecc3a49 None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80312) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:27:31.431359 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-6eb1b6c8-2d6b-47b9-8a21-b33e8ecc3a49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:27:31.497308 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 261/509] 10.222.0.57 () {58 vars in 1023 bytes} [Fri Jul 1 16:27:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 01 16:27:31.498152 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb1b6c8-2d6b-47b9-8a21-b33e8ecc3a49 None None] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:27:31.501620 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73891b9d-90fb-48d3-ad4e-819ebc5c51e0 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:27:31.540464 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c155f1c4-5511-4e11-9e4f-84e519d4bcd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:27:31.541120 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c155f1c4-5511-4e11-9e4f-84e519d4bcd6 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:31.541407 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c155f1c4-5511-4e11-9e4f-84e519d4bcd6 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:31.541689 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c155f1c4-5511-4e11-9e4f-84e519d4bcd6 None admin] PATH_INFO: `/v3/domains` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:31.542440 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c155f1c4-5511-4e11-9e4f-84e519d4bcd6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:27:31.542751 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c155f1c4-5511-4e11-9e4f-84e519d4bcd6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:27:31.544118 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c155f1c4-5511-4e11-9e4f-84e519d4bcd6 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:27:31.555671 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 249/510] 10.222.0.57 () {56 vars in 1109 bytes} [Fri Jul 1 16:27:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:27:31.556491 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c155f1c4-5511-4e11-9e4f-84e519d4bcd6 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:27:31.613860 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb1b6c8-2d6b-47b9-8a21-b33e8ecc3a49 None None] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:27:31.614578 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:27:31.643819 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=100 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:27:31.656143 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282c0df8-f430-4d54-bf7b-63812f5ed8ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:27:31.656887 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282c0df8-f430-4d54-bf7b-63812f5ed8ed None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:31.657239 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282c0df8-f430-4d54-bf7b-63812f5ed8ed None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:31.657574 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282c0df8-f430-4d54-bf7b-63812f5ed8ed None admin] PATH_INFO: `/v3/roles` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:31.658418 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-282c0df8-f430-4d54-bf7b-63812f5ed8ed None admin] RBAC: Adding query filter params (name=service) {{(pid=80312) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:27:31.658725 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-282c0df8-f430-4d54-bf7b-63812f5ed8ed None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:27:31.660082 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-282c0df8-f430-4d54-bf7b-63812f5ed8ed None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:27:31.660940 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c155f1c4-5511-4e11-9e4f-84e519d4bcd6 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:27:31.668605 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 262/511] 10.222.0.57 () {56 vars in 1103 bytes} [Fri Jul 1 16:27:31 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:27:31.669401 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282c0df8-f430-4d54-bf7b-63812f5ed8ed None admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:27:31.698028 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2478542-dcab-460e-82d6-9ac1320f9404 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:27:31.698899 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2478542-dcab-460e-82d6-9ac1320f9404 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:31.699210 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2478542-dcab-460e-82d6-9ac1320f9404 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:31.699481 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2478542-dcab-460e-82d6-9ac1320f9404 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:31.706327 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2478542-dcab-460e-82d6-9ac1320f9404 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:27:31.707789 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2478542-dcab-460e-82d6-9ac1320f9404 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:27:31.722870 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application [None req-b2478542-dcab-460e-82d6-9ac1320f9404 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 01 16:27:31.722870 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 16:27:31.722870 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 16:27:31.722870 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 16:27:31.722870 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 16:27:31.722870 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 16:27:31.722870 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 16:27:31.722870 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 16:27:31.722870 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 16:27:31.722870 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 16:27:31.722870 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 16:27:31.722870 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 16:27:31.722870 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 01 16:27:31.722870 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 01 16:27:31.722870 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 01 16:27:31.723886 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 01 16:27:31.723886 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 16:27:31.723886 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 16:27:31.723886 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 16:27:31.723886 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 16:27:31.723886 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 16:27:31.723886 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 16:27:31.723886 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 16:27:31.723886 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application with Lock( Jul 01 16:27:31.723886 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 16:27:31.723886 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._enter() Jul 01 16:27:31.723886 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 16:27:31.723886 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 16:27:31.723886 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 16:27:31.723886 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.creator() Jul 01 16:27:31.723886 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 16:27:31.723886 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application created_value = creator( Jul 01 16:27:31.724979 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 01 16:27:31.724979 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 01 16:27:31.724979 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 01 16:27:31.724979 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 01 16:27:31.724979 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 01 16:27:31.724979 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 01 16:27:31.724979 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 01 16:27:31.724979 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Jul 01 16:27:31.729541 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 250/512] 10.222.0.57 () {56 vars in 1111 bytes} [Fri Jul 1 16:27:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 16:27:31.730254 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2478542-dcab-460e-82d6-9ac1320f9404 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:27:31.769177 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282c0df8-f430-4d54-bf7b-63812f5ed8ed None admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:27:31.802758 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03265059-0357-4eaf-9517-4f597b93eb12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:27:31.803402 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03265059-0357-4eaf-9517-4f597b93eb12 None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:31.803685 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03265059-0357-4eaf-9517-4f597b93eb12 None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:31.803969 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03265059-0357-4eaf-9517-4f597b93eb12 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:31.829611 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2478542-dcab-460e-82d6-9ac1320f9404 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:27:31.850421 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:27:31.859999 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=100 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:27:31.865062 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10db75c7-5703-4226-8734-57905835b810 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:27:31.865852 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10db75c7-5703-4226-8734-57905835b810 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:31.866131 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10db75c7-5703-4226-8734-57905835b810 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:31.866433 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10db75c7-5703-4226-8734-57905835b810 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:31.876598 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10db75c7-5703-4226-8734-57905835b810 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:27:31.877116 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03265059-0357-4eaf-9517-4f597b93eb12 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:27:31.878038 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10db75c7-5703-4226-8734-57905835b810 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:27:31.878595 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03265059-0357-4eaf-9517-4f597b93eb12 None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:27:31.895792 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application [None req-10db75c7-5703-4226-8734-57905835b810 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 16:27:31.895792 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 16:27:31.895792 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 16:27:31.895792 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 16:27:31.895792 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 16:27:31.895792 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 16:27:31.895792 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 16:27:31.895792 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 16:27:31.895792 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 16:27:31.895792 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 16:27:31.895792 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 16:27:31.895792 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 16:27:31.895792 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 01 16:27:31.895792 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 01 16:27:31.895792 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 01 16:27:31.896629 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 01 16:27:31.896629 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 16:27:31.896629 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 16:27:31.896629 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 16:27:31.896629 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 16:27:31.896629 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 16:27:31.896629 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 16:27:31.896629 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 16:27:31.896629 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application with Lock( Jul 01 16:27:31.896629 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 16:27:31.896629 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._enter() Jul 01 16:27:31.896629 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 16:27:31.896629 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 16:27:31.896629 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 16:27:31.896629 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.creator() Jul 01 16:27:31.896629 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 16:27:31.896629 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application created_value = creator( Jul 01 16:27:31.897480 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 01 16:27:31.897480 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 01 16:27:31.897480 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 16:27:31.897480 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Jul 01 16:27:31.897480 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 251/513] 10.222.0.57 () {56 vars in 1108 bytes} [Fri Jul 1 16:27:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 16:27:31.897774 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10db75c7-5703-4226-8734-57905835b810 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:27:31.979304 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 263/514] 10.222.0.57 () {58 vars in 1282 bytes} [Fri Jul 1 16:27:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 01 16:27:31.980156 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03265059-0357-4eaf-9517-4f597b93eb12 None admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:27:31.998876 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10db75c7-5703-4226-8734-57905835b810 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:27:32.032514 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1defd667-a22c-4645-9500-3c75a40910d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:27:32.033292 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1defd667-a22c-4645-9500-3c75a40910d3 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:32.033576 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1defd667-a22c-4645-9500-3c75a40910d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:32.033854 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1defd667-a22c-4645-9500-3c75a40910d3 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:32.041229 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1defd667-a22c-4645-9500-3c75a40910d3 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:27:32.042699 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1defd667-a22c-4645-9500-3c75a40910d3 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:27:32.056739 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application [None req-1defd667-a22c-4645-9500-3c75a40910d3 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 01 16:27:32.056739 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 16:27:32.056739 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 16:27:32.056739 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 16:27:32.056739 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 16:27:32.056739 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 16:27:32.056739 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 16:27:32.056739 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 16:27:32.056739 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 16:27:32.056739 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 16:27:32.056739 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 16:27:32.056739 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 16:27:32.056739 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 01 16:27:32.056739 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 01 16:27:32.056739 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 01 16:27:32.056739 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 01 16:27:32.057655 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 16:27:32.057655 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 16:27:32.057655 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 01 16:27:32.057655 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 01 16:27:32.057655 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 01 16:27:32.057655 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 01 16:27:32.057655 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 16:27:32.057655 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 16:27:32.057655 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 16:27:32.057655 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 16:27:32.057655 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 16:27:32.057655 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application with Lock( Jul 01 16:27:32.057655 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 16:27:32.057655 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._enter() Jul 01 16:27:32.057655 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 16:27:32.057655 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 16:27:32.057655 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 16:27:32.057655 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.creator() Jul 01 16:27:32.058578 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 16:27:32.058578 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application created_value = creator( Jul 01 16:27:32.058578 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 01 16:27:32.058578 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 01 16:27:32.058578 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 01 16:27:32.058578 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 01 16:27:32.058578 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 01 16:27:32.058578 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 01 16:27:32.058578 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 01 16:27:32.058578 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Jul 01 16:27:32.058578 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 252/515] 10.222.0.57 () {56 vars in 1099 bytes} [Fri Jul 1 16:27:31 2022] GET /identity/v3/users/cinder => generated 84 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 16:27:32.059155 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1defd667-a22c-4645-9500-3c75a40910d3 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:27:32.061085 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=100 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:27:32.078294 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03265059-0357-4eaf-9517-4f597b93eb12 None admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:27:32.089605 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:27:32.111820 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5318d9ed-c48e-42c1-ac70-71283d76c90f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:27:32.112691 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5318d9ed-c48e-42c1-ac70-71283d76c90f None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:32.112973 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5318d9ed-c48e-42c1-ac70-71283d76c90f None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:32.113246 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5318d9ed-c48e-42c1-ac70-71283d76c90f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:32.123546 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5318d9ed-c48e-42c1-ac70-71283d76c90f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:27:32.124998 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5318d9ed-c48e-42c1-ac70-71283d76c90f None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:27:32.142725 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application [None req-5318d9ed-c48e-42c1-ac70-71283d76c90f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 16:27:32.142725 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 16:27:32.142725 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 16:27:32.142725 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 16:27:32.142725 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 16:27:32.142725 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 16:27:32.142725 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 16:27:32.142725 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 16:27:32.142725 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 16:27:32.142725 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 16:27:32.142725 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 16:27:32.142725 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 16:27:32.142725 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 01 16:27:32.142725 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 01 16:27:32.142725 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 01 16:27:32.143774 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 01 16:27:32.143774 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 16:27:32.143774 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 16:27:32.143774 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 16:27:32.143774 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 16:27:32.143774 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 16:27:32.143774 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 16:27:32.143774 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 16:27:32.143774 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application with Lock( Jul 01 16:27:32.143774 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 16:27:32.143774 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self._enter() Jul 01 16:27:32.143774 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 16:27:32.143774 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 16:27:32.143774 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 16:27:32.143774 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.creator() Jul 01 16:27:32.143774 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 16:27:32.143774 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application created_value = creator( Jul 01 16:27:32.144868 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 01 16:27:32.144868 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 01 16:27:32.144868 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 16:27:32.144868 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application Jul 01 16:27:32.144868 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 264/516] 10.222.0.57 () {56 vars in 1108 bytes} [Fri Jul 1 16:27:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 16:27:32.144868 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5318d9ed-c48e-42c1-ac70-71283d76c90f None admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:27:32.167153 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1defd667-a22c-4645-9500-3c75a40910d3 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:27:32.201025 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfacbe97-e0a7-4f2d-8f2d-768864d34951 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:27:32.201686 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfacbe97-e0a7-4f2d-8f2d-768864d34951 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:32.201971 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfacbe97-e0a7-4f2d-8f2d-768864d34951 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:32.202254 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfacbe97-e0a7-4f2d-8f2d-768864d34951 None admin] PATH_INFO: `/v3/projects` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:32.203075 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfacbe97-e0a7-4f2d-8f2d-768864d34951 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:27:32.203387 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfacbe97-e0a7-4f2d-8f2d-768864d34951 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:27:32.205097 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfacbe97-e0a7-4f2d-8f2d-768864d34951 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:27:32.223062 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 253/517] 10.222.0.57 () {56 vars in 1148 bytes} [Fri Jul 1 16:27:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:27:32.223878 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfacbe97-e0a7-4f2d-8f2d-768864d34951 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:27:32.244024 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5318d9ed-c48e-42c1-ac70-71283d76c90f None admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:27:32.278929 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a766387-98d6-4e03-a07c-c0272f054b7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:27:32.279583 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a766387-98d6-4e03-a07c-c0272f054b7c None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:32.279866 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a766387-98d6-4e03-a07c-c0272f054b7c None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:32.280139 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a766387-98d6-4e03-a07c-c0272f054b7c None admin] PATH_INFO: `/v3/domains` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:32.280854 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a766387-98d6-4e03-a07c-c0272f054b7c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80312) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:27:32.281160 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a766387-98d6-4e03-a07c-c0272f054b7c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:27:32.282545 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a766387-98d6-4e03-a07c-c0272f054b7c None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:27:32.285254 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=100 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:27:32.295710 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 265/518] 10.222.0.57 () {56 vars in 1109 bytes} [Fri Jul 1 16:27:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:27:32.301052 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a766387-98d6-4e03-a07c-c0272f054b7c None admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:27:32.323341 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:27:32.336617 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfacbe97-e0a7-4f2d-8f2d-768864d34951 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:27:32.373098 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870827c9-f05f-4708-983e-e1a0909e4177 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:27:32.373891 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870827c9-f05f-4708-983e-e1a0909e4177 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:32.374179 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870827c9-f05f-4708-983e-e1a0909e4177 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:32.374501 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870827c9-f05f-4708-983e-e1a0909e4177 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:32.385946 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-870827c9-f05f-4708-983e-e1a0909e4177 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:27:32.387440 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-870827c9-f05f-4708-983e-e1a0909e4177 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:27:32.404161 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a766387-98d6-4e03-a07c-c0272f054b7c None admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:27:32.405799 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application [None req-870827c9-f05f-4708-983e-e1a0909e4177 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 16:27:32.405799 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 16:27:32.405799 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 16:27:32.405799 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 16:27:32.405799 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 16:27:32.405799 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 16:27:32.405799 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 16:27:32.405799 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 16:27:32.405799 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 16:27:32.405799 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 16:27:32.405799 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 16:27:32.405799 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 16:27:32.405799 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 01 16:27:32.405799 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 01 16:27:32.405799 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 01 16:27:32.406843 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 01 16:27:32.406843 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 16:27:32.406843 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 16:27:32.406843 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 16:27:32.406843 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 16:27:32.406843 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 16:27:32.406843 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 16:27:32.406843 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 16:27:32.406843 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application with Lock( Jul 01 16:27:32.406843 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 16:27:32.406843 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._enter() Jul 01 16:27:32.406843 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 16:27:32.406843 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 16:27:32.406843 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 16:27:32.406843 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.creator() Jul 01 16:27:32.406843 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 16:27:32.406843 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application created_value = creator( Jul 01 16:27:32.407932 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 01 16:27:32.407932 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 01 16:27:32.407932 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 16:27:32.407932 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Jul 01 16:27:32.407932 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 254/519] 10.222.0.57 () {56 vars in 1108 bytes} [Fri Jul 1 16:27:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 16:27:32.408493 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870827c9-f05f-4708-983e-e1a0909e4177 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:27:32.440678 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98cb1eef-0003-49f5-a384-db7a43678d68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:27:32.441265 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cb1eef-0003-49f5-a384-db7a43678d68 None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:32.441550 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cb1eef-0003-49f5-a384-db7a43678d68 None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:32.441831 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cb1eef-0003-49f5-a384-db7a43678d68 None admin] PATH_INFO: `/v3/users` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:32.442715 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98cb1eef-0003-49f5-a384-db7a43678d68 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80312) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:27:32.443028 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98cb1eef-0003-49f5-a384-db7a43678d68 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:27:32.444483 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98cb1eef-0003-49f5-a384-db7a43678d68 None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:27:32.462697 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 266/520] 10.222.0.57 () {56 vars in 1137 bytes} [Fri Jul 1 16:27:32 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:27:32.463526 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98cb1eef-0003-49f5-a384-db7a43678d68 None admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:27:32.519087 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=100 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:27:32.520221 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870827c9-f05f-4708-983e-e1a0909e4177 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:27:32.555412 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff77b28-dd9f-4de0-bd7f-0c65746fd893 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:27:32.556157 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff77b28-dd9f-4de0-bd7f-0c65746fd893 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:32.556518 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff77b28-dd9f-4de0-bd7f-0c65746fd893 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:32.556790 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff77b28-dd9f-4de0-bd7f-0c65746fd893 None admin] PATH_INFO: `/v3/domains` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:32.557508 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff77b28-dd9f-4de0-bd7f-0c65746fd893 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:27:32.557812 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff77b28-dd9f-4de0-bd7f-0c65746fd893 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:27:32.559199 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff77b28-dd9f-4de0-bd7f-0c65746fd893 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:27:32.565777 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98cb1eef-0003-49f5-a384-db7a43678d68 None admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:27:32.567772 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:27:32.571802 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 255/521] 10.222.0.57 () {56 vars in 1109 bytes} [Fri Jul 1 16:27:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:27:32.572602 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff77b28-dd9f-4de0-bd7f-0c65746fd893 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:27:32.602939 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8d7f60-88c9-4118-ba13-0a437a51e7b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:27:32.603601 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8d7f60-88c9-4118-ba13-0a437a51e7b4 None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:32.603889 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8d7f60-88c9-4118-ba13-0a437a51e7b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:32.604167 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8d7f60-88c9-4118-ba13-0a437a51e7b4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:32.605004 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b8d7f60-88c9-4118-ba13-0a437a51e7b4 None admin] RBAC: Adding query filter params (role.id=e8809b67c9dc471daa1c2bd42c9a2baf, scope.project.id=e0788e6725a84e57a4174284c1845755, user.id=7a50f745b31941b7a7265c0fc852f48e) {{(pid=80312) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:27:32.605320 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b8d7f60-88c9-4118-ba13-0a437a51e7b4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:27:32.606819 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b8d7f60-88c9-4118-ba13-0a437a51e7b4 None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:27:32.614499 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 267/522] 10.222.0.57 () {56 vars in 1374 bytes} [Fri Jul 1 16:27:32 2022] GET /identity/v3/role_assignments?user.id=7a50f745b31941b7a7265c0fc852f48e&scope.project.id=e0788e6725a84e57a4174284c1845755&role.id=e8809b67c9dc471daa1c2bd42c9a2baf => generated 259 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:27:32.618217 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8d7f60-88c9-4118-ba13-0a437a51e7b4 None admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:27:32.712564 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff77b28-dd9f-4de0-bd7f-0c65746fd893 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:27:32.740026 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8d7f60-88c9-4118-ba13-0a437a51e7b4 None admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:27:32.754544 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79d37dd-2677-4516-a370-0d9decf466c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:27:32.755525 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79d37dd-2677-4516-a370-0d9decf466c9 None admin] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:32.756105 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79d37dd-2677-4516-a370-0d9decf466c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:32.756105 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79d37dd-2677-4516-a370-0d9decf466c9 None admin] PATH_INFO: `/v3/users` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:32.756809 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79d37dd-2677-4516-a370-0d9decf466c9 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:27:32.759133 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79d37dd-2677-4516-a370-0d9decf466c9 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:27:32.761263 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=100 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:27:32.783168 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 256/523] 10.222.0.57 () {60 vars in 1132 bytes} [Fri Jul 1 16:27:32 2022] POST /identity/v3/users => generated 247 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 16:27:32.783390 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bbfc51a-153a-4909-bc9f-0e839b8df9bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:27:32.784038 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79d37dd-2677-4516-a370-0d9decf466c9 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:27:32.784233 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbfc51a-153a-4909-bc9f-0e839b8df9bd None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:32.784605 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbfc51a-153a-4909-bc9f-0e839b8df9bd None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:32.784969 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbfc51a-153a-4909-bc9f-0e839b8df9bd None admin] PATH_INFO: `/v3/domains` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:32.785803 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bbfc51a-153a-4909-bc9f-0e839b8df9bd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80312) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:27:32.786116 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bbfc51a-153a-4909-bc9f-0e839b8df9bd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:27:32.787584 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bbfc51a-153a-4909-bc9f-0e839b8df9bd None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:27:32.809397 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 268/524] 10.222.0.57 () {56 vars in 1109 bytes} [Fri Jul 1 16:27:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:27:32.809397 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bbfc51a-153a-4909-bc9f-0e839b8df9bd None admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:27:32.899129 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:27:32.961379 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bbfc51a-153a-4909-bc9f-0e839b8df9bd None admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:27:32.963752 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79d37dd-2677-4516-a370-0d9decf466c9 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:27:33.007275 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd4d4b5-9233-4a21-aaac-7f24399f5f60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:27:33.008649 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd4d4b5-9233-4a21-aaac-7f24399f5f60 None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:33.008915 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd4d4b5-9233-4a21-aaac-7f24399f5f60 None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:33.009190 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd4d4b5-9233-4a21-aaac-7f24399f5f60 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:33.011313 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d01384-f39a-478b-b496-a720153ed6ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:27:33.011959 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d01384-f39a-478b-b496-a720153ed6ab None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:33.012238 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d01384-f39a-478b-b496-a720153ed6ab None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:33.012986 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d01384-f39a-478b-b496-a720153ed6ab None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:33.017752 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd4d4b5-9233-4a21-aaac-7f24399f5f60 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:27:33.020072 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd4d4b5-9233-4a21-aaac-7f24399f5f60 None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:27:33.037705 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application [None req-ecd4d4b5-9233-4a21-aaac-7f24399f5f60 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 01 16:27:33.037705 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 16:27:33.037705 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 16:27:33.037705 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 16:27:33.037705 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 16:27:33.037705 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 16:27:33.037705 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 16:27:33.037705 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 16:27:33.037705 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 16:27:33.037705 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 16:27:33.037705 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 16:27:33.037705 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 16:27:33.037705 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 01 16:27:33.037705 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 01 16:27:33.037705 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 01 16:27:33.037705 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 01 16:27:33.038779 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 16:27:33.038779 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 16:27:33.038779 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 01 16:27:33.038779 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 01 16:27:33.038779 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 01 16:27:33.038779 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 01 16:27:33.038779 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 16:27:33.038779 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 16:27:33.038779 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 16:27:33.038779 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 16:27:33.038779 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 16:27:33.038779 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application with Lock( Jul 01 16:27:33.038779 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 16:27:33.038779 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self._enter() Jul 01 16:27:33.038779 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 16:27:33.038779 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 16:27:33.038779 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 16:27:33.038779 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.creator() Jul 01 16:27:33.040408 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 16:27:33.040408 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application created_value = creator( Jul 01 16:27:33.040408 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 01 16:27:33.040408 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 01 16:27:33.040408 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 01 16:27:33.040408 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 01 16:27:33.040408 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 01 16:27:33.040408 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 01 16:27:33.040408 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 01 16:27:33.040408 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application Jul 01 16:27:33.040408 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 269/525] 10.222.0.57 () {56 vars in 1102 bytes} [Fri Jul 1 16:27:32 2022] GET /identity/v3/users/neutron => generated 85 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 16:27:33.040408 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd4d4b5-9233-4a21-aaac-7f24399f5f60 None admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:27:33.084633 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=100 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:27:33.105898 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d01384-f39a-478b-b496-a720153ed6ab None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:27:33.108219 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d01384-f39a-478b-b496-a720153ed6ab None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:27:33.160518 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd4d4b5-9233-4a21-aaac-7f24399f5f60 None admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:27:33.173826 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:27:33.203038 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf972b54-8393-4c1c-adc0-c5f0e6ed2819 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:27:33.204545 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf972b54-8393-4c1c-adc0-c5f0e6ed2819 None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:33.205015 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf972b54-8393-4c1c-adc0-c5f0e6ed2819 None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:33.205470 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf972b54-8393-4c1c-adc0-c5f0e6ed2819 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:33.214028 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf972b54-8393-4c1c-adc0-c5f0e6ed2819 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:27:33.216035 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf972b54-8393-4c1c-adc0-c5f0e6ed2819 None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:27:33.225463 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 257/526] 10.222.0.57 () {58 vars in 1282 bytes} [Fri Jul 1 16:27:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 442 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 01 16:27:33.226380 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d01384-f39a-478b-b496-a720153ed6ab None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:27:33.233499 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application [None req-cf972b54-8393-4c1c-adc0-c5f0e6ed2819 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 01 16:27:33.233499 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 16:27:33.233499 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 16:27:33.233499 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 16:27:33.233499 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 16:27:33.233499 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 16:27:33.233499 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 16:27:33.233499 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 16:27:33.233499 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 16:27:33.233499 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 16:27:33.233499 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 16:27:33.233499 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 16:27:33.233499 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 01 16:27:33.233499 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 01 16:27:33.233499 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 01 16:27:33.234680 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 01 16:27:33.234680 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 16:27:33.234680 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 16:27:33.234680 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 16:27:33.234680 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 16:27:33.234680 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 16:27:33.234680 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 16:27:33.234680 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 16:27:33.234680 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application with Lock( Jul 01 16:27:33.234680 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 16:27:33.234680 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self._enter() Jul 01 16:27:33.234680 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 16:27:33.234680 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 16:27:33.234680 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 16:27:33.234680 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.creator() Jul 01 16:27:33.234680 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 16:27:33.234680 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application created_value = creator( Jul 01 16:27:33.235562 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 01 16:27:33.235562 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 01 16:27:33.235562 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 01 16:27:33.235562 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 01 16:27:33.235562 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 01 16:27:33.235562 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 01 16:27:33.235562 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 01 16:27:33.235562 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application Jul 01 16:27:33.238090 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 270/527] 10.222.0.57 () {56 vars in 1111 bytes} [Fri Jul 1 16:27:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 16:27:33.239244 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf972b54-8393-4c1c-adc0-c5f0e6ed2819 None admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:27:33.362606 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=100 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:27:33.370566 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf972b54-8393-4c1c-adc0-c5f0e6ed2819 None admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:27:33.375132 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d01384-f39a-478b-b496-a720153ed6ab None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:27:33.415298 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81343d2e-7720-488c-8785-bd2869e32893 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:27:33.416399 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81343d2e-7720-488c-8785-bd2869e32893 None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:33.416873 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81343d2e-7720-488c-8785-bd2869e32893 None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:33.417465 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81343d2e-7720-488c-8785-bd2869e32893 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:33.426232 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48123fef-9f19-4b23-9d5e-a181f053a210 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:27:33.427148 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48123fef-9f19-4b23-9d5e-a181f053a210 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:33.427543 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48123fef-9f19-4b23-9d5e-a181f053a210 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:33.427921 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48123fef-9f19-4b23-9d5e-a181f053a210 None admin] PATH_INFO: `/v3/users` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:33.428909 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48123fef-9f19-4b23-9d5e-a181f053a210 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:27:33.430501 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48123fef-9f19-4b23-9d5e-a181f053a210 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:27:33.432775 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48123fef-9f19-4b23-9d5e-a181f053a210 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:27:33.434823 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81343d2e-7720-488c-8785-bd2869e32893 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:27:33.436455 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81343d2e-7720-488c-8785-bd2869e32893 None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:27:33.457192 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application [None req-81343d2e-7720-488c-8785-bd2869e32893 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 16:27:33.457192 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 16:27:33.457192 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 16:27:33.457192 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 16:27:33.457192 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 16:27:33.457192 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 16:27:33.457192 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 16:27:33.457192 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 16:27:33.457192 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 16:27:33.457192 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 16:27:33.457192 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 16:27:33.457192 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 16:27:33.457192 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 01 16:27:33.457192 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 01 16:27:33.457192 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 01 16:27:33.458143 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 01 16:27:33.458143 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 16:27:33.458143 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 16:27:33.458143 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 16:27:33.458143 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 16:27:33.458143 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 16:27:33.458143 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 16:27:33.458143 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 16:27:33.458143 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application with Lock( Jul 01 16:27:33.458143 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 16:27:33.458143 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self._enter() Jul 01 16:27:33.458143 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 16:27:33.458143 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 16:27:33.458143 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 16:27:33.458143 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.creator() Jul 01 16:27:33.458143 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 16:27:33.458143 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application created_value = creator( Jul 01 16:27:33.459233 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 01 16:27:33.459233 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 01 16:27:33.459233 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 16:27:33.459233 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application Jul 01 16:27:33.461600 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 271/528] 10.222.0.57 () {56 vars in 1108 bytes} [Fri Jul 1 16:27:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 16:27:33.462714 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81343d2e-7720-488c-8785-bd2869e32893 None admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:27:33.482490 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 258/529] 10.222.0.57 () {56 vars in 1139 bytes} [Fri Jul 1 16:27:33 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:27:33.484048 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48123fef-9f19-4b23-9d5e-a181f053a210 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:27:33.505193 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:27:33.612348 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81343d2e-7720-488c-8785-bd2869e32893 None admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:27:33.616335 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48123fef-9f19-4b23-9d5e-a181f053a210 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:27:33.657936 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5af778b-0cb4-48ab-82bb-34317ce9adf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:27:33.658634 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5af778b-0cb4-48ab-82bb-34317ce9adf1 None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:33.658900 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5af778b-0cb4-48ab-82bb-34317ce9adf1 None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:33.659175 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5af778b-0cb4-48ab-82bb-34317ce9adf1 None admin] PATH_INFO: `/v3/projects` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:33.659974 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5af778b-0cb4-48ab-82bb-34317ce9adf1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80312) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:27:33.660284 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5af778b-0cb4-48ab-82bb-34317ce9adf1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:27:33.661872 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5af778b-0cb4-48ab-82bb-34317ce9adf1 None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:27:33.664647 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693ff2c1-fed7-442e-abff-5bb547a166ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:27:33.665772 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693ff2c1-fed7-442e-abff-5bb547a166ba None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:33.666241 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693ff2c1-fed7-442e-abff-5bb547a166ba None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:33.667274 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693ff2c1-fed7-442e-abff-5bb547a166ba None admin] PATH_INFO: `/v3/domains` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:33.668205 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-693ff2c1-fed7-442e-abff-5bb547a166ba None admin] RBAC: Adding query filter params (name=Default) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:27:33.668694 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-693ff2c1-fed7-442e-abff-5bb547a166ba None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:27:33.670289 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-693ff2c1-fed7-442e-abff-5bb547a166ba None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:27:33.686422 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 272/530] 10.222.0.57 () {56 vars in 1148 bytes} [Fri Jul 1 16:27:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:27:33.686422 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5af778b-0cb4-48ab-82bb-34317ce9adf1 None admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:27:33.692978 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 259/531] 10.222.0.57 () {56 vars in 1109 bytes} [Fri Jul 1 16:27:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:27:33.698315 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693ff2c1-fed7-442e-abff-5bb547a166ba None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:27:33.701573 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=100 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:27:33.785251 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:27:33.817915 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693ff2c1-fed7-442e-abff-5bb547a166ba None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:27:33.840501 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5af778b-0cb4-48ab-82bb-34317ce9adf1 None admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:27:33.865942 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d081b1-a515-4566-afc4-12468c852895 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:27:33.867629 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d081b1-a515-4566-afc4-12468c852895 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:33.868114 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d081b1-a515-4566-afc4-12468c852895 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:33.868581 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d081b1-a515-4566-afc4-12468c852895 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:33.869611 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d081b1-a515-4566-afc4-12468c852895 None admin] RBAC: Adding query filter params (role.id=e8809b67c9dc471daa1c2bd42c9a2baf, scope.project.id=e0788e6725a84e57a4174284c1845755, user.id=fd028715bfea43099e7c6cde429ecf92) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:27:33.870205 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d081b1-a515-4566-afc4-12468c852895 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:27:33.872012 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d081b1-a515-4566-afc4-12468c852895 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:27:33.885967 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd384ea2-7b20-4532-8cf5-9d0247141d0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:27:33.885967 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd384ea2-7b20-4532-8cf5-9d0247141d0b None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:33.885967 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd384ea2-7b20-4532-8cf5-9d0247141d0b None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:33.885967 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd384ea2-7b20-4532-8cf5-9d0247141d0b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:33.886980 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 260/532] 10.222.0.57 () {56 vars in 1374 bytes} [Fri Jul 1 16:27:33 2022] GET /identity/v3/role_assignments?user.id=fd028715bfea43099e7c6cde429ecf92&scope.project.id=e0788e6725a84e57a4174284c1845755&role.id=e8809b67c9dc471daa1c2bd42c9a2baf => generated 259 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:27:33.888216 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d081b1-a515-4566-afc4-12468c852895 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:27:33.998294 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd384ea2-7b20-4532-8cf5-9d0247141d0b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:27:33.999814 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd384ea2-7b20-4532-8cf5-9d0247141d0b None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:27:34.014905 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=100 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:27:34.114400 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d081b1-a515-4566-afc4-12468c852895 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:27:34.167609 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56e2a94-2fc3-40d3-b2b8-40468f0a497c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:27:34.168772 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56e2a94-2fc3-40d3-b2b8-40468f0a497c None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:34.169252 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56e2a94-2fc3-40d3-b2b8-40468f0a497c None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:34.169722 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56e2a94-2fc3-40d3-b2b8-40468f0a497c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:34.178391 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b56e2a94-2fc3-40d3-b2b8-40468f0a497c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:27:34.181051 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b56e2a94-2fc3-40d3-b2b8-40468f0a497c None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:27:34.200586 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application [None req-b56e2a94-2fc3-40d3-b2b8-40468f0a497c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 01 16:27:34.200586 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 16:27:34.200586 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 16:27:34.200586 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 16:27:34.200586 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 16:27:34.200586 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 16:27:34.200586 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 16:27:34.200586 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 16:27:34.200586 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 16:27:34.200586 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 16:27:34.200586 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 16:27:34.200586 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 16:27:34.200586 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 01 16:27:34.200586 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 01 16:27:34.200586 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 01 16:27:34.201466 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 01 16:27:34.201466 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 16:27:34.201466 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 16:27:34.201466 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 16:27:34.201466 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 16:27:34.201466 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 16:27:34.201466 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 16:27:34.201466 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 16:27:34.201466 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application with Lock( Jul 01 16:27:34.201466 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 16:27:34.201466 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._enter() Jul 01 16:27:34.201466 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 16:27:34.201466 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 16:27:34.201466 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 16:27:34.201466 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.creator() Jul 01 16:27:34.201466 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 16:27:34.201466 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application created_value = creator( Jul 01 16:27:34.202309 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 01 16:27:34.202309 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 01 16:27:34.202309 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 01 16:27:34.202309 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 01 16:27:34.202309 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 01 16:27:34.202309 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 01 16:27:34.202309 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 01 16:27:34.202309 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Jul 01 16:27:34.203193 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 273/533] 10.222.0.57 () {58 vars in 1282 bytes} [Fri Jul 1 16:27:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 520 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 01 16:27:34.205096 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 261/534] 10.222.0.57 () {56 vars in 1111 bytes} [Fri Jul 1 16:27:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 317 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 16:27:34.211109 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:27:34.213850 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd384ea2-7b20-4532-8cf5-9d0247141d0b None admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:27:34.218309 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56e2a94-2fc3-40d3-b2b8-40468f0a497c None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:27:34.392492 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd384ea2-7b20-4532-8cf5-9d0247141d0b None admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:27:34.412091 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=100 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:27:34.417205 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56e2a94-2fc3-40d3-b2b8-40468f0a497c None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:27:34.458018 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef8adfb-0438-40a1-adb8-fc462e402c27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:27:34.459348 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef8adfb-0438-40a1-adb8-fc462e402c27 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:34.459831 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef8adfb-0438-40a1-adb8-fc462e402c27 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:34.460298 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef8adfb-0438-40a1-adb8-fc462e402c27 None admin] PATH_INFO: `/v3/projects` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:34.461328 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef8adfb-0438-40a1-adb8-fc462e402c27 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:27:34.461833 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef8adfb-0438-40a1-adb8-fc462e402c27 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:27:34.463520 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef8adfb-0438-40a1-adb8-fc462e402c27 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:27:34.472387 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c6a562-a62a-4f00-baaa-de0dc9c0541d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:27:34.473504 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c6a562-a62a-4f00-baaa-de0dc9c0541d None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:34.473990 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c6a562-a62a-4f00-baaa-de0dc9c0541d None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:34.474486 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c6a562-a62a-4f00-baaa-de0dc9c0541d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:34.487085 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 262/535] 10.222.0.57 () {56 vars in 1148 bytes} [Fri Jul 1 16:27:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 01 16:27:34.491146 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c6a562-a62a-4f00-baaa-de0dc9c0541d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:27:34.493491 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef8adfb-0438-40a1-adb8-fc462e402c27 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:27:34.495377 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c6a562-a62a-4f00-baaa-de0dc9c0541d None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:27:34.543243 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application [None req-62c6a562-a62a-4f00-baaa-de0dc9c0541d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 16:27:34.543243 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 16:27:34.543243 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 16:27:34.543243 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 16:27:34.543243 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 16:27:34.543243 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 16:27:34.543243 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 16:27:34.543243 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 16:27:34.543243 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 16:27:34.543243 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 16:27:34.543243 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 16:27:34.543243 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 16:27:34.543243 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 01 16:27:34.543243 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 01 16:27:34.543243 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 01 16:27:34.544158 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 01 16:27:34.544158 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 16:27:34.544158 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 16:27:34.544158 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 16:27:34.544158 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 16:27:34.544158 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 16:27:34.544158 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 16:27:34.544158 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 16:27:34.544158 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application with Lock( Jul 01 16:27:34.544158 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 16:27:34.544158 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self._enter() Jul 01 16:27:34.544158 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 16:27:34.544158 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 16:27:34.544158 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 16:27:34.544158 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.creator() Jul 01 16:27:34.544158 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 16:27:34.544158 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application created_value = creator( Jul 01 16:27:34.544994 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 01 16:27:34.544994 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 01 16:27:34.544994 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 16:27:34.544994 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application Jul 01 16:27:34.547391 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 274/536] 10.222.0.57 () {56 vars in 1108 bytes} [Fri Jul 1 16:27:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 334 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 16:27:34.552609 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c6a562-a62a-4f00-baaa-de0dc9c0541d None admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:27:34.604579 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:27:34.679024 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef8adfb-0438-40a1-adb8-fc462e402c27 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:27:34.715239 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c6a562-a62a-4f00-baaa-de0dc9c0541d None admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:27:34.751311 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c83b27c-c2a5-43c2-b5ef-488c22a7e0e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:27:34.753118 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c83b27c-c2a5-43c2-b5ef-488c22a7e0e3 None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:34.753405 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c83b27c-c2a5-43c2-b5ef-488c22a7e0e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:34.753682 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c83b27c-c2a5-43c2-b5ef-488c22a7e0e3 None admin] PATH_INFO: `/v3/domains` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:34.754436 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c83b27c-c2a5-43c2-b5ef-488c22a7e0e3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80312) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:27:34.754743 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c83b27c-c2a5-43c2-b5ef-488c22a7e0e3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:27:34.756122 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c83b27c-c2a5-43c2-b5ef-488c22a7e0e3 None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:27:34.771906 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad88c181-4357-4f33-81cc-75840a824112 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:27:34.772562 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad88c181-4357-4f33-81cc-75840a824112 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:34.772880 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad88c181-4357-4f33-81cc-75840a824112 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:34.773233 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad88c181-4357-4f33-81cc-75840a824112 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:34.774068 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad88c181-4357-4f33-81cc-75840a824112 None admin] RBAC: Adding query filter params (role.id=e8809b67c9dc471daa1c2bd42c9a2baf, scope.project.id=e0788e6725a84e57a4174284c1845755, user.id=dbddf971a2354bbdbb9da1d63f3abf88) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:27:34.774393 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad88c181-4357-4f33-81cc-75840a824112 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:27:34.776082 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad88c181-4357-4f33-81cc-75840a824112 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:27:34.783881 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 263/537] 10.222.0.57 () {56 vars in 1374 bytes} [Fri Jul 1 16:27:34 2022] GET /identity/v3/role_assignments?user.id=dbddf971a2354bbdbb9da1d63f3abf88&scope.project.id=e0788e6725a84e57a4174284c1845755&role.id=e8809b67c9dc471daa1c2bd42c9a2baf => generated 259 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 01 16:27:34.799175 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 275/538] 10.222.0.57 () {56 vars in 1109 bytes} [Fri Jul 1 16:27:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:27:34.803850 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad88c181-4357-4f33-81cc-75840a824112 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:27:34.858053 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef3bc91-e7ee-42a8-8b7e-0e1665196063 None None] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:34.858652 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef3bc91-e7ee-42a8-8b7e-0e1665196063 None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:34.858652 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef3bc91-e7ee-42a8-8b7e-0e1665196063 None None] PATH_INFO: `/` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:34.860338 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 276/539] 10.222.0.57 () {54 vars in 926 bytes} [Fri Jul 1 16:27:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 16:27:34.867343 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e4be99-0279-4154-a2d2-bea31525f111 None None] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:34.867952 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e4be99-0279-4154-a2d2-bea31525f111 None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:34.868457 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e4be99-0279-4154-a2d2-bea31525f111 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:34.887736 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.auth.core [None req-a4e4be99-0279-4154-a2d2-bea31525f111 None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80312) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:27:34.889403 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-a4e4be99-0279-4154-a2d2-bea31525f111 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:27:34.911647 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=100 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:27:35.016044 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 277/540] 10.222.0.57 () {58 vars in 1023 bytes} [Fri Jul 1 16:27:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 01 16:27:35.028478 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad88c181-4357-4f33-81cc-75840a824112 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:27:35.058316 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:27:35.077557 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e126c99-2140-4843-bda2-a94471ee6e6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:27:35.078709 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e126c99-2140-4843-bda2-a94471ee6e6a None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:35.079518 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e126c99-2140-4843-bda2-a94471ee6e6a None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:35.079997 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e126c99-2140-4843-bda2-a94471ee6e6a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:35.088345 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e126c99-2140-4843-bda2-a94471ee6e6a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:27:35.090096 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e126c99-2140-4843-bda2-a94471ee6e6a None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:27:35.109984 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application [None req-1e126c99-2140-4843-bda2-a94471ee6e6a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 01 16:27:35.109984 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 16:27:35.109984 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 16:27:35.109984 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 16:27:35.109984 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 16:27:35.109984 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 16:27:35.109984 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 16:27:35.109984 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 16:27:35.109984 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 16:27:35.109984 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 16:27:35.109984 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 16:27:35.109984 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 16:27:35.109984 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 01 16:27:35.109984 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 01 16:27:35.109984 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 01 16:27:35.110932 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 01 16:27:35.110932 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 16:27:35.110932 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 16:27:35.110932 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 16:27:35.110932 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 16:27:35.110932 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 16:27:35.110932 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 16:27:35.110932 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 16:27:35.110932 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application with Lock( Jul 01 16:27:35.110932 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 16:27:35.110932 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._enter() Jul 01 16:27:35.110932 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 16:27:35.110932 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 16:27:35.110932 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 16:27:35.110932 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.creator() Jul 01 16:27:35.110932 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 16:27:35.110932 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application created_value = creator( Jul 01 16:27:35.111770 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 01 16:27:35.111770 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 01 16:27:35.111770 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 01 16:27:35.111770 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 01 16:27:35.111770 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 01 16:27:35.111770 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 01 16:27:35.111770 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 01 16:27:35.111770 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Jul 01 16:27:35.115478 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 264/541] 10.222.0.57 () {56 vars in 1111 bytes} [Fri Jul 1 16:27:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 312 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 16:27:35.117755 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e4be99-0279-4154-a2d2-bea31525f111 None None] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:27:35.122049 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e126c99-2140-4843-bda2-a94471ee6e6a None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:27:35.233758 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e4be99-0279-4154-a2d2-bea31525f111 None None] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:27:35.242928 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e126c99-2140-4843-bda2-a94471ee6e6a None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:27:35.280289 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec64ba92-96f3-4762-9c9b-9145aca4b79d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:27:35.281103 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec64ba92-96f3-4762-9c9b-9145aca4b79d None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:35.281384 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec64ba92-96f3-4762-9c9b-9145aca4b79d None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:35.281658 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec64ba92-96f3-4762-9c9b-9145aca4b79d None admin] PATH_INFO: `/v3/roles/service` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:35.282768 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173e491d-03e1-4cc3-8dcb-77642a565df3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:27:35.283669 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173e491d-03e1-4cc3-8dcb-77642a565df3 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:35.284055 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173e491d-03e1-4cc3-8dcb-77642a565df3 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:35.284429 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173e491d-03e1-4cc3-8dcb-77642a565df3 None admin] PATH_INFO: `/v3/projects` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:35.285459 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-173e491d-03e1-4cc3-8dcb-77642a565df3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:27:35.285875 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-173e491d-03e1-4cc3-8dcb-77642a565df3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:27:35.287490 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-173e491d-03e1-4cc3-8dcb-77642a565df3 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:27:35.295966 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec64ba92-96f3-4762-9c9b-9145aca4b79d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 01 16:27:35.306397 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 265/542] 10.222.0.57 () {56 vars in 1148 bytes} [Fri Jul 1 16:27:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:27:35.306397 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b6eba0-2668-46ff-a832-c122794275bb None None] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:35.306397 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b6eba0-2668-46ff-a832-c122794275bb None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:35.306397 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b6eba0-2668-46ff-a832-c122794275bb None None] PATH_INFO: `/` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:35.309038 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 266/543] 10.222.0.57 () {54 vars in 926 bytes} [Fri Jul 1 16:27:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 16:27:35.310382 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae4ca8c-e943-4239-a11d-113f4255e178 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:35.310756 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae4ca8c-e943-4239-a11d-113f4255e178 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:35.311115 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae4ca8c-e943-4239-a11d-113f4255e178 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:35.312538 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application [None req-ec64ba92-96f3-4762-9c9b-9145aca4b79d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 01 16:27:35.312538 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 16:27:35.312538 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 16:27:35.312538 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 16:27:35.312538 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 16:27:35.312538 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 16:27:35.312538 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 16:27:35.312538 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 16:27:35.312538 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 16:27:35.312538 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 16:27:35.312538 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 16:27:35.312538 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 16:27:35.312538 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 01 16:27:35.312538 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 01 16:27:35.312538 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 01 16:27:35.312538 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 01 16:27:35.313432 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 01 16:27:35.313432 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 01 16:27:35.313432 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 01 16:27:35.313432 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 01 16:27:35.313432 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 16:27:35.313432 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 16:27:35.313432 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 16:27:35.313432 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 16:27:35.313432 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 16:27:35.313432 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 16:27:35.313432 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 16:27:35.313432 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application with Lock( Jul 01 16:27:35.313432 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 16:27:35.313432 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self._enter() Jul 01 16:27:35.313432 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 16:27:35.313432 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 16:27:35.314209 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 16:27:35.314209 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.creator() Jul 01 16:27:35.314209 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 16:27:35.314209 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application created_value = creator( Jul 01 16:27:35.314209 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 01 16:27:35.314209 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 01 16:27:35.314209 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 01 16:27:35.314209 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 01 16:27:35.314209 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 01 16:27:35.314209 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 01 16:27:35.314209 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 01 16:27:35.314209 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application Jul 01 16:27:35.317151 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 278/544] 10.222.0.57 () {56 vars in 1102 bytes} [Fri Jul 1 16:27:35 2022] GET /identity/v3/roles/service => generated 85 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 16:27:35.319243 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec64ba92-96f3-4762-9c9b-9145aca4b79d None admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:27:35.330934 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-bae4ca8c-e943-4239-a11d-113f4255e178 None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:27:35.332300 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-bae4ca8c-e943-4239-a11d-113f4255e178 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:27:35.338889 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=100 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:27:35.386240 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:27:35.432184 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 267/545] 10.222.0.57 () {58 vars in 1023 bytes} [Fri Jul 1 16:27:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 01 16:27:35.433804 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae4ca8c-e943-4239-a11d-113f4255e178 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:27:35.472223 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec64ba92-96f3-4762-9c9b-9145aca4b79d None admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:27:35.512560 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40929db-ad42-48d2-bb71-3139687b8d65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:27:35.513180 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40929db-ad42-48d2-bb71-3139687b8d65 None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:35.513467 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40929db-ad42-48d2-bb71-3139687b8d65 None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:35.513836 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40929db-ad42-48d2-bb71-3139687b8d65 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:35.515255 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f40929db-ad42-48d2-bb71-3139687b8d65 None admin] RBAC: Adding query filter params (role.id=e8809b67c9dc471daa1c2bd42c9a2baf, scope.project.id=e0788e6725a84e57a4174284c1845755, user.id=4ed200ce29684c5f80354fa9a8638fe2) {{(pid=80312) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:27:35.515255 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f40929db-ad42-48d2-bb71-3139687b8d65 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:27:35.516494 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f40929db-ad42-48d2-bb71-3139687b8d65 None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:27:35.522806 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 279/546] 10.222.0.57 () {56 vars in 1374 bytes} [Fri Jul 1 16:27:35 2022] GET /identity/v3/role_assignments?user.id=4ed200ce29684c5f80354fa9a8638fe2&scope.project.id=e0788e6725a84e57a4174284c1845755&role.id=e8809b67c9dc471daa1c2bd42c9a2baf => generated 259 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:27:35.527921 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40929db-ad42-48d2-bb71-3139687b8d65 None admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:27:35.563912 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae4ca8c-e943-4239-a11d-113f4255e178 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:27:35.612201 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=100 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:27:35.635625 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e19cf8-110f-42ba-b8f7-847851652588 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:27:35.636688 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e19cf8-110f-42ba-b8f7-847851652588 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:35.637198 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e19cf8-110f-42ba-b8f7-847851652588 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:35.637378 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40929db-ad42-48d2-bb71-3139687b8d65 None admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:27:35.637841 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e19cf8-110f-42ba-b8f7-847851652588 None admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:35.638908 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e19cf8-110f-42ba-b8f7-847851652588 None admin] RBAC: Adding query filter params (name=service) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:27:35.639409 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e19cf8-110f-42ba-b8f7-847851652588 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:27:35.641054 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e19cf8-110f-42ba-b8f7-847851652588 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:27:35.657101 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 268/547] 10.222.0.57 () {56 vars in 1103 bytes} [Fri Jul 1 16:27:35 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:27:35.662567 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e19cf8-110f-42ba-b8f7-847851652588 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:27:35.678702 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a4aeca-eac8-49ad-bdad-ade93723cbfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:27:35.680222 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a4aeca-eac8-49ad-bdad-ade93723cbfc None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:35.680222 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a4aeca-eac8-49ad-bdad-ade93723cbfc None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:35.680222 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a4aeca-eac8-49ad-bdad-ade93723cbfc None admin] PATH_INFO: `/v3/roles/service` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:35.692167 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a4aeca-eac8-49ad-bdad-ade93723cbfc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 01 16:27:35.706374 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application [None req-a3a4aeca-eac8-49ad-bdad-ade93723cbfc None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 01 16:27:35.706374 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 16:27:35.706374 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 16:27:35.706374 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 16:27:35.706374 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 16:27:35.706374 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 16:27:35.706374 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 16:27:35.706374 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 16:27:35.706374 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 16:27:35.706374 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 16:27:35.706374 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 16:27:35.706374 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 16:27:35.706374 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 01 16:27:35.706374 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 01 16:27:35.706374 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 01 16:27:35.706374 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 01 16:27:35.707349 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 01 16:27:35.707349 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 01 16:27:35.707349 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 01 16:27:35.707349 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 01 16:27:35.707349 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 16:27:35.707349 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 16:27:35.707349 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 16:27:35.707349 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 16:27:35.707349 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 16:27:35.707349 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 16:27:35.707349 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 16:27:35.707349 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application with Lock( Jul 01 16:27:35.707349 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 16:27:35.707349 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self._enter() Jul 01 16:27:35.707349 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 16:27:35.707349 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 16:27:35.708131 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 16:27:35.708131 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.creator() Jul 01 16:27:35.708131 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 16:27:35.708131 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application created_value = creator( Jul 01 16:27:35.708131 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 01 16:27:35.708131 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 01 16:27:35.708131 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 01 16:27:35.708131 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 01 16:27:35.708131 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 01 16:27:35.708131 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 01 16:27:35.708131 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 01 16:27:35.708131 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application Jul 01 16:27:35.708131 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 280/548] 10.222.0.57 () {56 vars in 1102 bytes} [Fri Jul 1 16:27:35 2022] GET /identity/v3/roles/service => generated 85 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 16:27:35.709372 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a4aeca-eac8-49ad-bdad-ade93723cbfc None admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:27:35.738846 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:27:35.862317 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a4aeca-eac8-49ad-bdad-ade93723cbfc None admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:27:35.864969 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e19cf8-110f-42ba-b8f7-847851652588 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:27:35.918742 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7da4098-6724-40ee-a56f-1091120c163b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:27:35.919536 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7da4098-6724-40ee-a56f-1091120c163b None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:35.919820 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7da4098-6724-40ee-a56f-1091120c163b None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:35.920094 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7da4098-6724-40ee-a56f-1091120c163b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:35.933047 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9741c9a-2013-4e03-9e45-5cd5bb541f4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:27:35.933727 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9741c9a-2013-4e03-9e45-5cd5bb541f4f None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:35.934010 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9741c9a-2013-4e03-9e45-5cd5bb541f4f None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:35.934288 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9741c9a-2013-4e03-9e45-5cd5bb541f4f None admin] PATH_INFO: `/v3/roles` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:35.935079 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9741c9a-2013-4e03-9e45-5cd5bb541f4f None admin] RBAC: Adding query filter params (name=service) {{(pid=80312) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:27:35.935390 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9741c9a-2013-4e03-9e45-5cd5bb541f4f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:27:35.936800 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9741c9a-2013-4e03-9e45-5cd5bb541f4f None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:27:35.942196 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=100 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:27:35.948350 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7da4098-6724-40ee-a56f-1091120c163b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:27:35.949825 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7da4098-6724-40ee-a56f-1091120c163b None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:27:35.959271 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 281/549] 10.222.0.57 () {56 vars in 1103 bytes} [Fri Jul 1 16:27:35 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:27:35.965199 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9741c9a-2013-4e03-9e45-5cd5bb541f4f None admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:27:35.987654 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application [None req-c7da4098-6724-40ee-a56f-1091120c163b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 16:27:35.987654 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 16:27:35.987654 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 16:27:35.987654 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 16:27:35.987654 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 16:27:35.987654 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 16:27:35.987654 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 16:27:35.987654 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 16:27:35.987654 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 16:27:35.987654 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 16:27:35.987654 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 16:27:35.987654 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 16:27:35.987654 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 01 16:27:35.987654 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 01 16:27:35.987654 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 01 16:27:35.989144 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 01 16:27:35.989144 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 16:27:35.989144 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 16:27:35.989144 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 16:27:35.989144 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 16:27:35.989144 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 16:27:35.989144 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 16:27:35.989144 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 16:27:35.989144 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application with Lock( Jul 01 16:27:35.989144 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 16:27:35.989144 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._enter() Jul 01 16:27:35.989144 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 16:27:35.989144 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 16:27:35.989144 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 16:27:35.989144 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.creator() Jul 01 16:27:35.989144 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 16:27:35.989144 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application created_value = creator( Jul 01 16:27:35.990080 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 01 16:27:35.990080 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 01 16:27:35.990080 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 16:27:35.990080 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Jul 01 16:27:35.990080 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 269/550] 10.222.0.57 () {56 vars in 1108 bytes} [Fri Jul 1 16:27:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 327 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 16:27:35.995987 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7da4098-6724-40ee-a56f-1091120c163b None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:27:36.152664 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9741c9a-2013-4e03-9e45-5cd5bb541f4f None admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:27:36.173498 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:27:36.202706 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7da4098-6724-40ee-a56f-1091120c163b None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:27:36.218755 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51c2152-68d4-43fb-9538-fc7ff5875881 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:27:36.218755 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51c2152-68d4-43fb-9538-fc7ff5875881 None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:36.218755 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51c2152-68d4-43fb-9538-fc7ff5875881 None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:36.218755 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51c2152-68d4-43fb-9538-fc7ff5875881 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:36.239789 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c51c2152-68d4-43fb-9538-fc7ff5875881 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:27:36.241279 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c51c2152-68d4-43fb-9538-fc7ff5875881 None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:27:36.248635 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce8bc65-981d-431f-aa59-334d24b93b31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:27:36.249659 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce8bc65-981d-431f-aa59-334d24b93b31 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:36.250140 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce8bc65-981d-431f-aa59-334d24b93b31 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:36.250643 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce8bc65-981d-431f-aa59-334d24b93b31 None admin] PATH_INFO: `/v3/domains` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:36.251585 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce8bc65-981d-431f-aa59-334d24b93b31 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:27:36.252099 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce8bc65-981d-431f-aa59-334d24b93b31 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:27:36.253704 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce8bc65-981d-431f-aa59-334d24b93b31 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:27:36.268660 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 270/551] 10.222.0.57 () {56 vars in 1109 bytes} [Fri Jul 1 16:27:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:27:36.276661 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application [None req-c51c2152-68d4-43fb-9538-fc7ff5875881 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 16:27:36.276661 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 16:27:36.276661 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 16:27:36.276661 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 16:27:36.276661 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 16:27:36.276661 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 16:27:36.276661 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 16:27:36.276661 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 16:27:36.276661 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 16:27:36.276661 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 16:27:36.276661 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 16:27:36.276661 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 16:27:36.276661 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 01 16:27:36.276661 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 01 16:27:36.276661 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 01 16:27:36.277523 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 01 16:27:36.277523 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 16:27:36.277523 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 16:27:36.277523 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 16:27:36.277523 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 16:27:36.277523 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 16:27:36.277523 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 16:27:36.277523 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 16:27:36.277523 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application with Lock( Jul 01 16:27:36.277523 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 16:27:36.277523 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self._enter() Jul 01 16:27:36.277523 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 16:27:36.277523 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 16:27:36.277523 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 16:27:36.277523 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.creator() Jul 01 16:27:36.277523 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 16:27:36.277523 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application created_value = creator( Jul 01 16:27:36.279036 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 01 16:27:36.279036 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 01 16:27:36.279036 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 16:27:36.279036 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application Jul 01 16:27:36.279036 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 282/552] 10.222.0.57 () {56 vars in 1108 bytes} [Fri Jul 1 16:27:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 313 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 16:27:36.280382 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce8bc65-981d-431f-aa59-334d24b93b31 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:27:36.283430 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51c2152-68d4-43fb-9538-fc7ff5875881 None admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:27:36.341454 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=100 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:27:36.396331 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51c2152-68d4-43fb-9538-fc7ff5875881 None admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:27:36.433744 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57af856e-aa8b-4a8a-8479-f1fbf129c5f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:27:36.434334 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57af856e-aa8b-4a8a-8479-f1fbf129c5f3 None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:36.434675 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57af856e-aa8b-4a8a-8479-f1fbf129c5f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:36.434919 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57af856e-aa8b-4a8a-8479-f1fbf129c5f3 None admin] PATH_INFO: `/v3/domains` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:36.435649 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57af856e-aa8b-4a8a-8479-f1fbf129c5f3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80312) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:27:36.435958 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57af856e-aa8b-4a8a-8479-f1fbf129c5f3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:27:36.437447 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57af856e-aa8b-4a8a-8479-f1fbf129c5f3 None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:27:36.447670 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce8bc65-981d-431f-aa59-334d24b93b31 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:27:36.451109 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 283/553] 10.222.0.57 () {56 vars in 1109 bytes} [Fri Jul 1 16:27:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:27:36.452414 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88dddb3f-60a8-4ce8-a0d9-f1dff02acc75 None None] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:36.452704 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88dddb3f-60a8-4ce8-a0d9-f1dff02acc75 None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:36.452984 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88dddb3f-60a8-4ce8-a0d9-f1dff02acc75 None None] PATH_INFO: `/` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:36.462391 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 284/554] 10.222.0.57 () {54 vars in 926 bytes} [Fri Jul 1 16:27:36 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 16:27:36.462391 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88dddb3f-60a8-4ce8-a0d9-f1dff02acc75 None None] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:27:36.491549 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c8fd4e-1b84-49e2-b10d-138cbaa41cf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:27:36.492827 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c8fd4e-1b84-49e2-b10d-138cbaa41cf4 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:36.493418 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c8fd4e-1b84-49e2-b10d-138cbaa41cf4 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:36.493999 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c8fd4e-1b84-49e2-b10d-138cbaa41cf4 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:36.503046 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c8fd4e-1b84-49e2-b10d-138cbaa41cf4 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:27:36.505335 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c8fd4e-1b84-49e2-b10d-138cbaa41cf4 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:27:36.523766 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application [None req-d3c8fd4e-1b84-49e2-b10d-138cbaa41cf4 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 01 16:27:36.523766 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 16:27:36.523766 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 16:27:36.523766 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 16:27:36.523766 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 16:27:36.523766 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 16:27:36.523766 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 16:27:36.523766 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 16:27:36.523766 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 16:27:36.523766 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 16:27:36.523766 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 16:27:36.523766 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 16:27:36.523766 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 01 16:27:36.523766 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 01 16:27:36.523766 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 01 16:27:36.523766 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 01 16:27:36.524840 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 16:27:36.524840 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 16:27:36.524840 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 01 16:27:36.524840 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 01 16:27:36.524840 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 01 16:27:36.524840 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 01 16:27:36.524840 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 16:27:36.524840 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 16:27:36.524840 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 16:27:36.524840 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 16:27:36.524840 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 16:27:36.524840 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application with Lock( Jul 01 16:27:36.524840 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 16:27:36.524840 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._enter() Jul 01 16:27:36.524840 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 16:27:36.524840 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 16:27:36.524840 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 16:27:36.524840 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.creator() Jul 01 16:27:36.525748 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 16:27:36.525748 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application created_value = creator( Jul 01 16:27:36.525748 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 01 16:27:36.525748 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 01 16:27:36.525748 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 01 16:27:36.525748 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 01 16:27:36.525748 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 01 16:27:36.525748 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 01 16:27:36.525748 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 01 16:27:36.525748 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Jul 01 16:27:36.533211 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 271/555] 10.222.0.57 () {56 vars in 1093 bytes} [Fri Jul 1 16:27:36 2022] GET /identity/v3/users/nova => generated 82 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 16:27:36.534933 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a931d6c8-0ded-4e75-b489-f5f187e02f4e None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:36.535442 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a931d6c8-0ded-4e75-b489-f5f187e02f4e None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:36.535906 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a931d6c8-0ded-4e75-b489-f5f187e02f4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:36.548238 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:27:36.555527 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-a931d6c8-0ded-4e75-b489-f5f187e02f4e None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:27:36.557177 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-a931d6c8-0ded-4e75-b489-f5f187e02f4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:27:36.580491 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88dddb3f-60a8-4ce8-a0d9-f1dff02acc75 None None] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:27:36.593566 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=100 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:27:36.623154 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed281fca-e140-4270-a7bd-78f0d56fffc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:27:36.624087 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed281fca-e140-4270-a7bd-78f0d56fffc2 None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:36.624345 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed281fca-e140-4270-a7bd-78f0d56fffc2 None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:36.624624 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed281fca-e140-4270-a7bd-78f0d56fffc2 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:36.633560 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed281fca-e140-4270-a7bd-78f0d56fffc2 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:27:36.635235 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed281fca-e140-4270-a7bd-78f0d56fffc2 None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:27:36.636573 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 272/556] 10.222.0.57 () {58 vars in 1023 bytes} [Fri Jul 1 16:27:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 01 16:27:36.643471 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a931d6c8-0ded-4e75-b489-f5f187e02f4e None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:27:36.652765 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application [None req-ed281fca-e140-4270-a7bd-78f0d56fffc2 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 01 16:27:36.652765 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 16:27:36.652765 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 16:27:36.652765 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 16:27:36.652765 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 16:27:36.652765 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 16:27:36.652765 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 16:27:36.652765 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 16:27:36.652765 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 16:27:36.652765 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 16:27:36.652765 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 16:27:36.652765 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 16:27:36.652765 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 01 16:27:36.652765 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 01 16:27:36.652765 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 01 16:27:36.652765 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 01 16:27:36.653962 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 16:27:36.653962 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 16:27:36.653962 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 01 16:27:36.653962 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 01 16:27:36.653962 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 01 16:27:36.653962 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 01 16:27:36.653962 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 16:27:36.653962 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 16:27:36.653962 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 16:27:36.653962 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 16:27:36.653962 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 16:27:36.653962 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application with Lock( Jul 01 16:27:36.653962 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 16:27:36.653962 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self._enter() Jul 01 16:27:36.653962 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 16:27:36.653962 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 16:27:36.653962 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 16:27:36.653962 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.creator() Jul 01 16:27:36.655606 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 16:27:36.655606 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application created_value = creator( Jul 01 16:27:36.655606 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 01 16:27:36.655606 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 01 16:27:36.655606 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 01 16:27:36.655606 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 01 16:27:36.655606 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 01 16:27:36.655606 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 01 16:27:36.655606 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 01 16:27:36.655606 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application Jul 01 16:27:36.658974 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 285/557] 10.222.0.57 () {56 vars in 1096 bytes} [Fri Jul 1 16:27:36 2022] GET /identity/v3/users/swift => generated 83 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 16:27:36.659910 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed281fca-e140-4270-a7bd-78f0d56fffc2 None admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:27:36.753392 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a931d6c8-0ded-4e75-b489-f5f187e02f4e None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:27:36.773585 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed281fca-e140-4270-a7bd-78f0d56fffc2 None admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:27:36.789972 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f018f7b-b645-4939-8fa4-bbd4c862db28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:27:36.790828 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f018f7b-b645-4939-8fa4-bbd4c862db28 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:36.791214 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f018f7b-b645-4939-8fa4-bbd4c862db28 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:36.791593 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f018f7b-b645-4939-8fa4-bbd4c862db28 None admin] PATH_INFO: `/v3/users` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:36.792548 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f018f7b-b645-4939-8fa4-bbd4c862db28 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:27:36.792988 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f018f7b-b645-4939-8fa4-bbd4c862db28 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:27:36.794568 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f018f7b-b645-4939-8fa4-bbd4c862db28 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:27:36.798677 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:27:36.810373 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe26e86-2171-4577-bc5a-0eb8cc5e28b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:27:36.811409 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe26e86-2171-4577-bc5a-0eb8cc5e28b0 None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:36.811699 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe26e86-2171-4577-bc5a-0eb8cc5e28b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:36.811976 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe26e86-2171-4577-bc5a-0eb8cc5e28b0 None admin] PATH_INFO: `/v3/users` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:36.812864 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe26e86-2171-4577-bc5a-0eb8cc5e28b0 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80312) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:27:36.813177 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe26e86-2171-4577-bc5a-0eb8cc5e28b0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:27:36.814243 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 273/558] 10.222.0.57 () {56 vars in 1133 bytes} [Fri Jul 1 16:27:36 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:27:36.814964 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe26e86-2171-4577-bc5a-0eb8cc5e28b0 None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:27:36.819464 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f018f7b-b645-4939-8fa4-bbd4c862db28 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:27:36.835264 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 286/559] 10.222.0.57 () {56 vars in 1135 bytes} [Fri Jul 1 16:27:36 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:27:36.842811 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe26e86-2171-4577-bc5a-0eb8cc5e28b0 None admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:27:36.871240 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=100 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:27:36.938119 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f018f7b-b645-4939-8fa4-bbd4c862db28 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:27:36.979810 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe26e86-2171-4577-bc5a-0eb8cc5e28b0 None admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:27:36.990214 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8178bb89-5cd6-46f4-8b98-f19fe79c7822 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:27:36.992366 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8178bb89-5cd6-46f4-8b98-f19fe79c7822 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:36.993236 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8178bb89-5cd6-46f4-8b98-f19fe79c7822 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:36.993690 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8178bb89-5cd6-46f4-8b98-f19fe79c7822 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:37.009641 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8178bb89-5cd6-46f4-8b98-f19fe79c7822 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 01 16:27:37.021093 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application [None req-8178bb89-5cd6-46f4-8b98-f19fe79c7822 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 01 16:27:37.021093 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 16:27:37.021093 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 16:27:37.021093 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 16:27:37.021093 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 16:27:37.021093 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 16:27:37.021093 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 16:27:37.021093 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 16:27:37.021093 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 16:27:37.021093 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 16:27:37.021093 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 16:27:37.021093 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 16:27:37.021093 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 01 16:27:37.021093 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 01 16:27:37.021093 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 01 16:27:37.021093 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 01 16:27:37.022335 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 01 16:27:37.022335 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 01 16:27:37.022335 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 01 16:27:37.022335 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 01 16:27:37.022335 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 16:27:37.022335 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 16:27:37.022335 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 16:27:37.022335 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 16:27:37.022335 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 16:27:37.022335 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 16:27:37.022335 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 16:27:37.022335 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application with Lock( Jul 01 16:27:37.022335 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 16:27:37.022335 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._enter() Jul 01 16:27:37.022335 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 16:27:37.022335 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 16:27:37.023866 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 16:27:37.023866 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.creator() Jul 01 16:27:37.023866 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 16:27:37.023866 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application created_value = creator( Jul 01 16:27:37.023866 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 01 16:27:37.023866 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 01 16:27:37.023866 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 01 16:27:37.023866 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 01 16:27:37.023866 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 01 16:27:37.023866 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 01 16:27:37.023866 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 01 16:27:37.023866 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Jul 01 16:27:37.025893 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c97d4f-f73f-45d9-85f7-d60bfecc17d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:27:37.026723 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 274/560] 10.222.0.57 () {56 vars in 1102 bytes} [Fri Jul 1 16:27:36 2022] GET /identity/v3/roles/service => generated 85 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 16:27:37.027559 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c97d4f-f73f-45d9-85f7-d60bfecc17d3 None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:37.027956 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8178bb89-5cd6-46f4-8b98-f19fe79c7822 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:27:37.028191 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c97d4f-f73f-45d9-85f7-d60bfecc17d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:37.028860 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c97d4f-f73f-45d9-85f7-d60bfecc17d3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:37.042223 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c97d4f-f73f-45d9-85f7-d60bfecc17d3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:27:37.043860 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c97d4f-f73f-45d9-85f7-d60bfecc17d3 None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:27:37.065857 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application [None req-79c97d4f-f73f-45d9-85f7-d60bfecc17d3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 16:27:37.065857 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 16:27:37.065857 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 16:27:37.065857 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 16:27:37.065857 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 16:27:37.065857 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 16:27:37.065857 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 16:27:37.065857 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 16:27:37.065857 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 16:27:37.065857 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 16:27:37.065857 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 16:27:37.065857 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 16:27:37.065857 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 01 16:27:37.065857 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 01 16:27:37.065857 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 01 16:27:37.067104 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 01 16:27:37.067104 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 16:27:37.067104 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 16:27:37.067104 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 16:27:37.067104 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 16:27:37.067104 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 16:27:37.067104 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 16:27:37.067104 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 16:27:37.067104 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application with Lock( Jul 01 16:27:37.067104 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 16:27:37.067104 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self._enter() Jul 01 16:27:37.067104 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 16:27:37.067104 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 16:27:37.067104 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 16:27:37.067104 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.creator() Jul 01 16:27:37.067104 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 16:27:37.067104 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application created_value = creator( Jul 01 16:27:37.068106 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 01 16:27:37.068106 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 01 16:27:37.068106 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 16:27:37.068106 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application Jul 01 16:27:37.068106 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 287/561] 10.222.0.57 () {56 vars in 1108 bytes} [Fri Jul 1 16:27:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 16:27:37.073153 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c97d4f-f73f-45d9-85f7-d60bfecc17d3 None admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:27:37.099470 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:27:37.152761 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8178bb89-5cd6-46f4-8b98-f19fe79c7822 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:27:37.179688 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=100 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:27:37.193678 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c957e45-ea2d-48a3-8d0c-3a738a0d3168 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:27:37.194332 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c957e45-ea2d-48a3-8d0c-3a738a0d3168 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:37.194702 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c957e45-ea2d-48a3-8d0c-3a738a0d3168 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:37.195016 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c957e45-ea2d-48a3-8d0c-3a738a0d3168 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:37.200573 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c97d4f-f73f-45d9-85f7-d60bfecc17d3 None admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:27:37.236747 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090002a2-0dac-4145-9feb-dd91b08f5fff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:27:37.237547 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090002a2-0dac-4145-9feb-dd91b08f5fff None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:37.237928 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090002a2-0dac-4145-9feb-dd91b08f5fff None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:37.238302 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090002a2-0dac-4145-9feb-dd91b08f5fff None admin] PATH_INFO: `/v3/roles` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:37.239166 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-090002a2-0dac-4145-9feb-dd91b08f5fff None admin] RBAC: Adding query filter params (name=service) {{(pid=80312) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:27:37.239572 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-090002a2-0dac-4145-9feb-dd91b08f5fff None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:27:37.241056 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-090002a2-0dac-4145-9feb-dd91b08f5fff None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:27:37.250477 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 288/562] 10.222.0.57 () {56 vars in 1103 bytes} [Fri Jul 1 16:27:37 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:27:37.251641 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd162890-0321-40ca-8ddf-f28c1590ec3c None None] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:37.252029 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd162890-0321-40ca-8ddf-f28c1590ec3c None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:37.252405 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd162890-0321-40ca-8ddf-f28c1590ec3c None None] PATH_INFO: `/` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:37.253718 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 289/563] 10.222.0.57 () {54 vars in 926 bytes} [Fri Jul 1 16:27:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 16:27:37.258623 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd162890-0321-40ca-8ddf-f28c1590ec3c None None] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:27:37.275607 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c957e45-ea2d-48a3-8d0c-3a738a0d3168 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:27:37.276994 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c957e45-ea2d-48a3-8d0c-3a738a0d3168 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:27:37.339816 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:27:37.376844 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd162890-0321-40ca-8ddf-f28c1590ec3c None None] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:27:37.389692 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 275/564] 10.222.0.57 () {58 vars in 1282 bytes} [Fri Jul 1 16:27:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 362 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 01 16:27:37.390584 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c957e45-ea2d-48a3-8d0c-3a738a0d3168 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:27:37.420804 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d66915-d749-476b-99e5-d6d374c3d96c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:27:37.421462 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d66915-d749-476b-99e5-d6d374c3d96c None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:37.421749 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d66915-d749-476b-99e5-d6d374c3d96c None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:37.422027 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d66915-d749-476b-99e5-d6d374c3d96c None admin] PATH_INFO: `/v3/domains` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:37.422777 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d66915-d749-476b-99e5-d6d374c3d96c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80312) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:27:37.423083 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d66915-d749-476b-99e5-d6d374c3d96c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:27:37.424506 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d66915-d749-476b-99e5-d6d374c3d96c None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:27:37.441141 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 290/565] 10.222.0.57 () {56 vars in 1109 bytes} [Fri Jul 1 16:27:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:27:37.441141 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c31124-9115-4b60-9cc9-8c867bcc157c None None] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:37.441141 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c31124-9115-4b60-9cc9-8c867bcc157c None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:37.441141 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c31124-9115-4b60-9cc9-8c867bcc157c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:37.443677 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=100 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:27:37.458619 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.auth.core [None req-c6c31124-9115-4b60-9cc9-8c867bcc157c None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80312) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:27:37.459893 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-c6c31124-9115-4b60-9cc9-8c867bcc157c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:27:37.505202 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c957e45-ea2d-48a3-8d0c-3a738a0d3168 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:27:37.531755 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 291/566] 10.222.0.57 () {58 vars in 1023 bytes} [Fri Jul 1 16:27:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 01 16:27:37.532528 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c31124-9115-4b60-9cc9-8c867bcc157c None None] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:27:37.560256 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f873c79-ce60-49ae-8763-cae257c6f78f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:27:37.561029 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f873c79-ce60-49ae-8763-cae257c6f78f None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:37.561314 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f873c79-ce60-49ae-8763-cae257c6f78f None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:37.561590 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f873c79-ce60-49ae-8763-cae257c6f78f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:37.573267 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f873c79-ce60-49ae-8763-cae257c6f78f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:27:37.574781 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f873c79-ce60-49ae-8763-cae257c6f78f None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:27:37.593157 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application [None req-1f873c79-ce60-49ae-8763-cae257c6f78f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 16:27:37.593157 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 16:27:37.593157 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 16:27:37.593157 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 16:27:37.593157 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 16:27:37.593157 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 16:27:37.593157 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 16:27:37.593157 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 16:27:37.593157 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 16:27:37.593157 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 16:27:37.593157 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 16:27:37.593157 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 16:27:37.593157 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 01 16:27:37.593157 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 01 16:27:37.593157 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 01 16:27:37.594263 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 01 16:27:37.594263 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 16:27:37.594263 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 16:27:37.594263 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 16:27:37.594263 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 16:27:37.594263 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 16:27:37.594263 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 16:27:37.594263 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 16:27:37.594263 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application with Lock( Jul 01 16:27:37.594263 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 16:27:37.594263 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._enter() Jul 01 16:27:37.594263 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 16:27:37.594263 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 16:27:37.594263 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 16:27:37.594263 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.creator() Jul 01 16:27:37.594263 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 16:27:37.594263 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application created_value = creator( Jul 01 16:27:37.599939 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 01 16:27:37.599939 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 01 16:27:37.599939 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 16:27:37.599939 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Jul 01 16:27:37.599939 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 276/567] 10.222.0.57 () {56 vars in 1108 bytes} [Fri Jul 1 16:27:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 16:27:37.599939 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f873c79-ce60-49ae-8763-cae257c6f78f None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:27:37.607793 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:27:37.650588 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c31124-9115-4b60-9cc9-8c867bcc157c None None] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:27:37.698087 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=100 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:27:37.707085 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6476b869-ebee-463d-820d-60d13db13fef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:27:37.708502 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6476b869-ebee-463d-820d-60d13db13fef None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:37.709070 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6476b869-ebee-463d-820d-60d13db13fef None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:37.709521 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6476b869-ebee-463d-820d-60d13db13fef None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:37.724640 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6476b869-ebee-463d-820d-60d13db13fef None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:27:37.726028 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6476b869-ebee-463d-820d-60d13db13fef None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:27:37.727319 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f873c79-ce60-49ae-8763-cae257c6f78f None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:27:37.748865 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application [None req-6476b869-ebee-463d-820d-60d13db13fef None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 16:27:37.748865 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 16:27:37.748865 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 16:27:37.748865 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 16:27:37.748865 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 16:27:37.748865 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 16:27:37.748865 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 16:27:37.748865 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 16:27:37.748865 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 16:27:37.748865 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 16:27:37.748865 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 16:27:37.748865 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 16:27:37.748865 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 01 16:27:37.748865 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 01 16:27:37.748865 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 01 16:27:37.749988 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 01 16:27:37.749988 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 16:27:37.749988 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 16:27:37.749988 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 16:27:37.749988 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 16:27:37.749988 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 16:27:37.749988 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 16:27:37.749988 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 16:27:37.749988 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application with Lock( Jul 01 16:27:37.749988 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 16:27:37.749988 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self._enter() Jul 01 16:27:37.749988 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 16:27:37.749988 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 16:27:37.749988 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 16:27:37.749988 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.creator() Jul 01 16:27:37.749988 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 16:27:37.749988 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application created_value = creator( Jul 01 16:27:37.750924 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 01 16:27:37.750924 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 01 16:27:37.750924 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 16:27:37.750924 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application Jul 01 16:27:37.750924 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 292/568] 10.222.0.57 () {56 vars in 1108 bytes} [Fri Jul 1 16:27:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 16:27:37.750924 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6476b869-ebee-463d-820d-60d13db13fef None admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:27:37.774539 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749d123f-f80a-414d-9f3a-a69ecce89674 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:27:37.775336 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749d123f-f80a-414d-9f3a-a69ecce89674 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:37.775616 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749d123f-f80a-414d-9f3a-a69ecce89674 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:37.775889 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749d123f-f80a-414d-9f3a-a69ecce89674 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:37.783419 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749d123f-f80a-414d-9f3a-a69ecce89674 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:27:37.784831 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749d123f-f80a-414d-9f3a-a69ecce89674 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:27:37.799861 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application [None req-749d123f-f80a-414d-9f3a-a69ecce89674 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 01 16:27:37.799861 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 16:27:37.799861 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 16:27:37.799861 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 16:27:37.799861 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 16:27:37.799861 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 16:27:37.799861 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 16:27:37.799861 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 16:27:37.799861 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 16:27:37.799861 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 16:27:37.799861 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 16:27:37.799861 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 16:27:37.799861 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 01 16:27:37.799861 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 01 16:27:37.799861 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 01 16:27:37.800844 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 01 16:27:37.800844 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 16:27:37.800844 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 16:27:37.800844 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 16:27:37.800844 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 16:27:37.800844 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 16:27:37.800844 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 16:27:37.800844 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 16:27:37.800844 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application with Lock( Jul 01 16:27:37.800844 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 16:27:37.800844 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._enter() Jul 01 16:27:37.800844 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 16:27:37.800844 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 16:27:37.800844 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 16:27:37.800844 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.creator() Jul 01 16:27:37.800844 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 16:27:37.800844 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application created_value = creator( Jul 01 16:27:37.801717 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 01 16:27:37.801717 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 01 16:27:37.801717 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 01 16:27:37.801717 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 01 16:27:37.801717 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 01 16:27:37.801717 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 01 16:27:37.801717 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 01 16:27:37.801717 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Jul 01 16:27:37.806425 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 277/569] 10.222.0.57 () {56 vars in 1111 bytes} [Fri Jul 1 16:27:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 16:27:37.807152 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749d123f-f80a-414d-9f3a-a69ecce89674 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:27:37.874759 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6476b869-ebee-463d-820d-60d13db13fef None admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:27:37.883990 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:27:37.915878 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f56a282-8a8f-4e71-ba0e-2402e46cafcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:27:37.917189 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f56a282-8a8f-4e71-ba0e-2402e46cafcc None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:37.917550 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f56a282-8a8f-4e71-ba0e-2402e46cafcc None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:37.917831 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f56a282-8a8f-4e71-ba0e-2402e46cafcc None admin] PATH_INFO: `/v3/roles/service` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:37.920133 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749d123f-f80a-414d-9f3a-a69ecce89674 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:27:37.928651 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f56a282-8a8f-4e71-ba0e-2402e46cafcc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 01 16:27:37.937931 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application [None req-1f56a282-8a8f-4e71-ba0e-2402e46cafcc None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 01 16:27:37.937931 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 16:27:37.937931 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 16:27:37.937931 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 16:27:37.937931 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 16:27:37.937931 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 16:27:37.937931 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 16:27:37.937931 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 16:27:37.937931 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 16:27:37.937931 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 16:27:37.937931 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 16:27:37.937931 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 16:27:37.937931 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 01 16:27:37.937931 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 01 16:27:37.937931 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 01 16:27:37.937931 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 01 16:27:37.939335 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 01 16:27:37.939335 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 01 16:27:37.939335 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 01 16:27:37.939335 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 01 16:27:37.939335 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 16:27:37.939335 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 16:27:37.939335 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 16:27:37.939335 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 16:27:37.939335 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 16:27:37.939335 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 16:27:37.939335 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 16:27:37.939335 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application with Lock( Jul 01 16:27:37.939335 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 16:27:37.939335 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self._enter() Jul 01 16:27:37.939335 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 16:27:37.939335 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 16:27:37.940256 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 16:27:37.940256 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.creator() Jul 01 16:27:37.940256 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 16:27:37.940256 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application created_value = creator( Jul 01 16:27:37.940256 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 01 16:27:37.940256 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 01 16:27:37.940256 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 01 16:27:37.940256 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 01 16:27:37.940256 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 01 16:27:37.940256 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 01 16:27:37.940256 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 01 16:27:37.940256 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application Jul 01 16:27:37.940256 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 293/570] 10.222.0.57 () {56 vars in 1102 bytes} [Fri Jul 1 16:27:37 2022] GET /identity/v3/roles/service => generated 85 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 16:27:37.940256 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f56a282-8a8f-4e71-ba0e-2402e46cafcc None admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:27:37.959935 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7868b45c-746d-4207-bcd4-2a52adc92652 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:27:37.960727 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7868b45c-746d-4207-bcd4-2a52adc92652 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:37.961525 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7868b45c-746d-4207-bcd4-2a52adc92652 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:37.961873 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7868b45c-746d-4207-bcd4-2a52adc92652 None admin] PATH_INFO: `/v3/domains` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:37.962638 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7868b45c-746d-4207-bcd4-2a52adc92652 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:27:37.962950 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7868b45c-746d-4207-bcd4-2a52adc92652 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:27:37.964319 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7868b45c-746d-4207-bcd4-2a52adc92652 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:27:37.980641 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 278/571] 10.222.0.57 () {56 vars in 1109 bytes} [Fri Jul 1 16:27:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:27:37.980641 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7868b45c-746d-4207-bcd4-2a52adc92652 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:27:37.994154 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=100 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:27:38.048792 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f56a282-8a8f-4e71-ba0e-2402e46cafcc None admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:27:38.080770 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7868b45c-746d-4207-bcd4-2a52adc92652 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:27:38.082388 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ec0a3d-b5d1-4d85-a1e5-644b15cb0686 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:27:38.083061 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ec0a3d-b5d1-4d85-a1e5-644b15cb0686 None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:38.083350 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ec0a3d-b5d1-4d85-a1e5-644b15cb0686 None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:38.083633 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ec0a3d-b5d1-4d85-a1e5-644b15cb0686 None admin] PATH_INFO: `/v3/domains` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:38.084361 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ec0a3d-b5d1-4d85-a1e5-644b15cb0686 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80312) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:27:38.084673 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ec0a3d-b5d1-4d85-a1e5-644b15cb0686 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:27:38.086124 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ec0a3d-b5d1-4d85-a1e5-644b15cb0686 None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:27:38.097971 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 294/572] 10.222.0.57 () {56 vars in 1109 bytes} [Fri Jul 1 16:27:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:27:38.098849 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ec0a3d-b5d1-4d85-a1e5-644b15cb0686 None admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:27:38.108528 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:27:38.119100 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00dabf6-2d7a-4ad7-8896-4051b617d6c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:27:38.119766 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00dabf6-2d7a-4ad7-8896-4051b617d6c9 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:38.120051 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00dabf6-2d7a-4ad7-8896-4051b617d6c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:38.120331 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00dabf6-2d7a-4ad7-8896-4051b617d6c9 None admin] PATH_INFO: `/v3/projects` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:38.121124 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b00dabf6-2d7a-4ad7-8896-4051b617d6c9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:27:38.121437 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b00dabf6-2d7a-4ad7-8896-4051b617d6c9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:27:38.122906 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b00dabf6-2d7a-4ad7-8896-4051b617d6c9 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:27:38.135312 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 279/573] 10.222.0.57 () {56 vars in 1148 bytes} [Fri Jul 1 16:27:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:27:38.136114 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00dabf6-2d7a-4ad7-8896-4051b617d6c9 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:27:38.200104 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ec0a3d-b5d1-4d85-a1e5-644b15cb0686 None admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:27:38.202326 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=100 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:27:38.236883 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2496a0ea-682e-4ef7-8538-53e2b7d79bcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:27:38.237546 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2496a0ea-682e-4ef7-8538-53e2b7d79bcd None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:38.237825 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2496a0ea-682e-4ef7-8538-53e2b7d79bcd None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:38.238100 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2496a0ea-682e-4ef7-8538-53e2b7d79bcd None admin] PATH_INFO: `/v3/roles` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:38.238292 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00dabf6-2d7a-4ad7-8896-4051b617d6c9 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:27:38.238879 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2496a0ea-682e-4ef7-8538-53e2b7d79bcd None admin] RBAC: Adding query filter params (name=service) {{(pid=80312) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:27:38.239187 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2496a0ea-682e-4ef7-8538-53e2b7d79bcd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:27:38.240548 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2496a0ea-682e-4ef7-8538-53e2b7d79bcd None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:27:38.248936 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 295/574] 10.222.0.57 () {56 vars in 1103 bytes} [Fri Jul 1 16:27:38 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:27:38.250157 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa16e5a-b432-4027-914a-1885a11fc372 None None] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:38.250479 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa16e5a-b432-4027-914a-1885a11fc372 None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:38.250752 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa16e5a-b432-4027-914a-1885a11fc372 None None] PATH_INFO: `/` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:38.253570 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 296/575] 10.222.0.57 () {54 vars in 926 bytes} [Fri Jul 1 16:27:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 16:27:38.253570 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa16e5a-b432-4027-914a-1885a11fc372 None None] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:27:38.276044 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5044779-268c-40b1-bd06-cbb9f94a27ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:27:38.276845 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5044779-268c-40b1-bd06-cbb9f94a27ad None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:38.277131 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5044779-268c-40b1-bd06-cbb9f94a27ad None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:38.277411 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5044779-268c-40b1-bd06-cbb9f94a27ad None admin] PATH_INFO: `/v3/users/glance` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:38.284634 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5044779-268c-40b1-bd06-cbb9f94a27ad None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:27:38.286086 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5044779-268c-40b1-bd06-cbb9f94a27ad None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:27:38.301921 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application [None req-f5044779-268c-40b1-bd06-cbb9f94a27ad None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 01 16:27:38.301921 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 16:27:38.301921 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 16:27:38.301921 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 16:27:38.301921 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 16:27:38.301921 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 16:27:38.301921 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 16:27:38.301921 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 16:27:38.301921 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 16:27:38.301921 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 16:27:38.301921 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 16:27:38.301921 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 16:27:38.301921 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 01 16:27:38.301921 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 01 16:27:38.301921 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 01 16:27:38.301921 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 01 16:27:38.302970 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 16:27:38.302970 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 16:27:38.302970 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 01 16:27:38.302970 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 01 16:27:38.302970 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 01 16:27:38.302970 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 01 16:27:38.302970 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 16:27:38.302970 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 16:27:38.302970 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 16:27:38.302970 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 16:27:38.302970 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 16:27:38.302970 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application with Lock( Jul 01 16:27:38.302970 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 16:27:38.302970 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._enter() Jul 01 16:27:38.302970 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 16:27:38.302970 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 16:27:38.302970 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 16:27:38.302970 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.creator() Jul 01 16:27:38.305260 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 16:27:38.305260 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application created_value = creator( Jul 01 16:27:38.305260 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 01 16:27:38.305260 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 01 16:27:38.305260 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 01 16:27:38.305260 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 01 16:27:38.305260 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 01 16:27:38.305260 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 01 16:27:38.305260 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 01 16:27:38.305260 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Jul 01 16:27:38.305260 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 280/576] 10.222.0.57 () {56 vars in 1099 bytes} [Fri Jul 1 16:27:38 2022] GET /identity/v3/users/glance => generated 84 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 16:27:38.305260 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5044779-268c-40b1-bd06-cbb9f94a27ad None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:27:38.352753 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:27:38.364408 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa16e5a-b432-4027-914a-1885a11fc372 None None] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:27:38.403224 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d83918-a3af-4e82-9adb-5fb2a4a9d508 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:27:38.404022 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d83918-a3af-4e82-9adb-5fb2a4a9d508 None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:38.404407 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d83918-a3af-4e82-9adb-5fb2a4a9d508 None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:38.404583 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d83918-a3af-4e82-9adb-5fb2a4a9d508 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:38.407866 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5044779-268c-40b1-bd06-cbb9f94a27ad None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:27:38.411249 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d83918-a3af-4e82-9adb-5fb2a4a9d508 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:27:38.412704 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d83918-a3af-4e82-9adb-5fb2a4a9d508 None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:27:38.426429 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application [None req-b4d83918-a3af-4e82-9adb-5fb2a4a9d508 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 01 16:27:38.426429 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 16:27:38.426429 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 16:27:38.426429 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 16:27:38.426429 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 16:27:38.426429 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 16:27:38.426429 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 16:27:38.426429 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 16:27:38.426429 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 16:27:38.426429 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 16:27:38.426429 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 16:27:38.426429 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 16:27:38.426429 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 01 16:27:38.426429 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 01 16:27:38.426429 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 01 16:27:38.427523 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 01 16:27:38.427523 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 16:27:38.427523 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 16:27:38.427523 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 16:27:38.427523 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 16:27:38.427523 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 16:27:38.427523 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 16:27:38.427523 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 16:27:38.427523 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application with Lock( Jul 01 16:27:38.427523 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 16:27:38.427523 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self._enter() Jul 01 16:27:38.427523 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 16:27:38.427523 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 16:27:38.427523 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 16:27:38.427523 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.creator() Jul 01 16:27:38.427523 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 16:27:38.427523 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application created_value = creator( Jul 01 16:27:38.428816 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 01 16:27:38.428816 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 01 16:27:38.428816 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 01 16:27:38.428816 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 01 16:27:38.428816 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 01 16:27:38.428816 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 01 16:27:38.428816 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 01 16:27:38.428816 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application Jul 01 16:27:38.428816 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 297/577] 10.222.0.57 () {56 vars in 1111 bytes} [Fri Jul 1 16:27:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 16:27:38.428816 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d83918-a3af-4e82-9adb-5fb2a4a9d508 None admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:27:38.447559 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ddf833b-4d56-4d08-b834-1e115927876d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:27:38.448439 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddf833b-4d56-4d08-b834-1e115927876d None admin] REQUEST_METHOD: `PUT` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:38.448717 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddf833b-4d56-4d08-b834-1e115927876d None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:38.448997 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddf833b-4d56-4d08-b834-1e115927876d None admin] PATH_INFO: `/v3/projects/e0788e6725a84e57a4174284c1845755/users/7a50f745b31941b7a7265c0fc852f48e/roles/e8809b67c9dc471daa1c2bd42c9a2baf` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:38.457631 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=100 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:27:38.463520 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ddf833b-4d56-4d08-b834-1e115927876d None admin] RBAC: Authorizing `identity:create_grant(project_id=e0788e6725a84e57a4174284c1845755, user_id=7a50f745b31941b7a7265c0fc852f48e, role_id=e8809b67c9dc471daa1c2bd42c9a2baf)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:27:38.464948 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ddf833b-4d56-4d08-b834-1e115927876d None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:27:38.477224 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 281/578] 10.222.0.57 () {58 vars in 1439 bytes} [Fri Jul 1 16:27:38 2022] PUT /identity/v3/projects/e0788e6725a84e57a4174284c1845755/users/7a50f745b31941b7a7265c0fc852f48e/roles/e8809b67c9dc471daa1c2bd42c9a2baf => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 16:27:38.478480 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a47fde9-f146-404f-b17f-cbded88fc34e None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:38.478770 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a47fde9-f146-404f-b17f-cbded88fc34e None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:38.479302 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a47fde9-f146-404f-b17f-cbded88fc34e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:38.504082 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-5a47fde9-f146-404f-b17f-cbded88fc34e None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:27:38.505337 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-5a47fde9-f146-404f-b17f-cbded88fc34e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:27:38.544332 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d83918-a3af-4e82-9adb-5fb2a4a9d508 None admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:27:38.583748 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f6bfb7-7fa1-454f-a025-2608e1007654 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:27:38.584917 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f6bfb7-7fa1-454f-a025-2608e1007654 None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:38.585028 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 282/579] 10.222.0.57 () {58 vars in 1023 bytes} [Fri Jul 1 16:27:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 01 16:27:38.586219 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f6bfb7-7fa1-454f-a025-2608e1007654 None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:38.586729 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a47fde9-f146-404f-b17f-cbded88fc34e None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:27:38.587134 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f6bfb7-7fa1-454f-a025-2608e1007654 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:38.607704 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f6bfb7-7fa1-454f-a025-2608e1007654 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:27:38.609844 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f6bfb7-7fa1-454f-a025-2608e1007654 None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:27:38.623916 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:27:38.633492 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application [None req-f4f6bfb7-7fa1-454f-a025-2608e1007654 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 16:27:38.633492 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 16:27:38.633492 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 16:27:38.633492 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 16:27:38.633492 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 16:27:38.633492 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 16:27:38.633492 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 16:27:38.633492 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 16:27:38.633492 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 16:27:38.633492 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 16:27:38.633492 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 16:27:38.633492 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 16:27:38.633492 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 01 16:27:38.633492 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 01 16:27:38.633492 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 01 16:27:38.634932 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 01 16:27:38.634932 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 16:27:38.634932 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 16:27:38.634932 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 16:27:38.634932 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 16:27:38.634932 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 16:27:38.634932 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 16:27:38.634932 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 16:27:38.634932 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application with Lock( Jul 01 16:27:38.634932 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 16:27:38.634932 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self._enter() Jul 01 16:27:38.634932 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 16:27:38.634932 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 16:27:38.634932 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 16:27:38.634932 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.creator() Jul 01 16:27:38.634932 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 16:27:38.634932 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application created_value = creator( Jul 01 16:27:38.636228 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 01 16:27:38.636228 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 01 16:27:38.636228 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 16:27:38.636228 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application Jul 01 16:27:38.636228 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 298/580] 10.222.0.57 () {56 vars in 1108 bytes} [Fri Jul 1 16:27:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 16:27:38.636228 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f6bfb7-7fa1-454f-a025-2608e1007654 None admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:27:38.718538 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a47fde9-f146-404f-b17f-cbded88fc34e None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:27:38.754513 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=100 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:27:38.758434 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a224fcd6-dbd9-4c9c-bcc5-a4b07f6c9e59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:27:38.759112 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a224fcd6-dbd9-4c9c-bcc5-a4b07f6c9e59 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:38.759394 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a224fcd6-dbd9-4c9c-bcc5-a4b07f6c9e59 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:38.759668 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a224fcd6-dbd9-4c9c-bcc5-a4b07f6c9e59 None admin] PATH_INFO: `/v3/users` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:38.760522 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a224fcd6-dbd9-4c9c-bcc5-a4b07f6c9e59 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:27:38.760831 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a224fcd6-dbd9-4c9c-bcc5-a4b07f6c9e59 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:27:38.762276 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a224fcd6-dbd9-4c9c-bcc5-a4b07f6c9e59 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:27:38.780668 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f6bfb7-7fa1-454f-a025-2608e1007654 None admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:27:38.786810 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 283/581] 10.222.0.57 () {56 vars in 1137 bytes} [Fri Jul 1 16:27:38 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:27:38.786810 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a224fcd6-dbd9-4c9c-bcc5-a4b07f6c9e59 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:27:38.816718 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38974434-e148-4360-ae52-46e64b09fc9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:27:38.817407 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38974434-e148-4360-ae52-46e64b09fc9b None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:38.817690 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38974434-e148-4360-ae52-46e64b09fc9b None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:38.817966 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38974434-e148-4360-ae52-46e64b09fc9b None admin] PATH_INFO: `/v3/projects` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:38.818797 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38974434-e148-4360-ae52-46e64b09fc9b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80312) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:27:38.819103 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38974434-e148-4360-ae52-46e64b09fc9b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:27:38.821297 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38974434-e148-4360-ae52-46e64b09fc9b None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:27:38.839530 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 299/582] 10.222.0.57 () {56 vars in 1148 bytes} [Fri Jul 1 16:27:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:27:38.839530 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38974434-e148-4360-ae52-46e64b09fc9b None admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:27:38.903201 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:27:38.926502 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a224fcd6-dbd9-4c9c-bcc5-a4b07f6c9e59 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:27:38.964507 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38974434-e148-4360-ae52-46e64b09fc9b None admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:27:38.978626 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3498a408-9574-48c5-851b-8b5b65133aaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:27:38.979267 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3498a408-9574-48c5-851b-8b5b65133aaf None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:38.979557 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3498a408-9574-48c5-851b-8b5b65133aaf None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:38.979844 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3498a408-9574-48c5-851b-8b5b65133aaf None admin] PATH_INFO: `/v3/roles/service` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:38.994684 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3498a408-9574-48c5-851b-8b5b65133aaf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 01 16:27:39.002165 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application [None req-3498a408-9574-48c5-851b-8b5b65133aaf None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 01 16:27:39.002165 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 16:27:39.002165 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 16:27:39.002165 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 16:27:39.002165 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 16:27:39.002165 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 16:27:39.002165 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 16:27:39.002165 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 16:27:39.002165 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 16:27:39.002165 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 16:27:39.002165 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 16:27:39.002165 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 16:27:39.002165 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 01 16:27:39.002165 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 01 16:27:39.002165 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 01 16:27:39.002165 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 01 16:27:39.003394 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 01 16:27:39.003394 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 01 16:27:39.003394 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 01 16:27:39.003394 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 01 16:27:39.003394 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 16:27:39.003394 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 16:27:39.003394 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 16:27:39.003394 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 16:27:39.003394 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 16:27:39.003394 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 16:27:39.003394 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 16:27:39.003394 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application with Lock( Jul 01 16:27:39.003394 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 16:27:39.003394 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._enter() Jul 01 16:27:39.003394 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 16:27:39.003394 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 16:27:39.004253 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 16:27:39.004253 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.creator() Jul 01 16:27:39.004253 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 16:27:39.004253 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application created_value = creator( Jul 01 16:27:39.004253 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 01 16:27:39.004253 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 01 16:27:39.004253 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 01 16:27:39.004253 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 01 16:27:39.004253 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 01 16:27:39.004253 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 01 16:27:39.004253 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 01 16:27:39.004253 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Jul 01 16:27:39.004253 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 284/583] 10.222.0.57 () {56 vars in 1102 bytes} [Fri Jul 1 16:27:38 2022] GET /identity/v3/roles/service => generated 85 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 16:27:39.004253 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3498a408-9574-48c5-851b-8b5b65133aaf None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:27:39.023279 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293109b5-6499-4353-96bd-4f0a28716103 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:27:39.024306 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293109b5-6499-4353-96bd-4f0a28716103 None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:39.024306 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293109b5-6499-4353-96bd-4f0a28716103 None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:39.024609 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293109b5-6499-4353-96bd-4f0a28716103 None admin] PATH_INFO: `/v3/domains` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:39.025239 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293109b5-6499-4353-96bd-4f0a28716103 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80312) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:27:39.025547 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293109b5-6499-4353-96bd-4f0a28716103 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:27:39.026967 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293109b5-6499-4353-96bd-4f0a28716103 None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:27:39.035141 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=100 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:27:39.046434 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 300/584] 10.222.0.57 () {56 vars in 1109 bytes} [Fri Jul 1 16:27:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:27:39.047367 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293109b5-6499-4353-96bd-4f0a28716103 None admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:27:39.154495 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3498a408-9574-48c5-851b-8b5b65133aaf None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:27:39.183036 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293109b5-6499-4353-96bd-4f0a28716103 None admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:27:39.196280 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58666e69-c3e7-443a-8e51-0c2219b4f0ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:27:39.197099 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58666e69-c3e7-443a-8e51-0c2219b4f0ff None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:39.197675 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58666e69-c3e7-443a-8e51-0c2219b4f0ff None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:39.197675 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58666e69-c3e7-443a-8e51-0c2219b4f0ff None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:39.212292 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58666e69-c3e7-443a-8e51-0c2219b4f0ff None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:27:39.213935 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58666e69-c3e7-443a-8e51-0c2219b4f0ff None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:27:39.217530 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:27:39.227907 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0831f262-c17d-456c-9509-5bcc78121126 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:27:39.228568 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0831f262-c17d-456c-9509-5bcc78121126 None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:39.228854 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0831f262-c17d-456c-9509-5bcc78121126 None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:39.229123 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0831f262-c17d-456c-9509-5bcc78121126 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:39.229990 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0831f262-c17d-456c-9509-5bcc78121126 None admin] RBAC: Adding query filter params (role.id=e8809b67c9dc471daa1c2bd42c9a2baf, scope.project.id=e0788e6725a84e57a4174284c1845755, user.id=859fe410c2d2450fb803c6244839f27c) {{(pid=80312) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:27:39.230297 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0831f262-c17d-456c-9509-5bcc78121126 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:27:39.231749 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0831f262-c17d-456c-9509-5bcc78121126 None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:27:39.236474 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application [None req-58666e69-c3e7-443a-8e51-0c2219b4f0ff None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 16:27:39.236474 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 16:27:39.236474 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 16:27:39.236474 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 16:27:39.236474 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 16:27:39.236474 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 16:27:39.236474 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 16:27:39.236474 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 16:27:39.236474 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 16:27:39.236474 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 16:27:39.236474 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 16:27:39.236474 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 16:27:39.236474 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 01 16:27:39.236474 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 01 16:27:39.236474 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 01 16:27:39.237825 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 01 16:27:39.237825 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 16:27:39.237825 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 16:27:39.237825 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 16:27:39.237825 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 16:27:39.237825 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 16:27:39.237825 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 16:27:39.237825 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 16:27:39.237825 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application with Lock( Jul 01 16:27:39.237825 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 16:27:39.237825 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._enter() Jul 01 16:27:39.237825 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 16:27:39.237825 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 16:27:39.237825 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 16:27:39.237825 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.creator() Jul 01 16:27:39.237825 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 16:27:39.237825 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application created_value = creator( Jul 01 16:27:39.239203 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 01 16:27:39.239203 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 01 16:27:39.239203 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 16:27:39.239203 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Jul 01 16:27:39.243878 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 285/585] 10.222.0.57 () {56 vars in 1108 bytes} [Fri Jul 1 16:27:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 16:27:39.244879 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58666e69-c3e7-443a-8e51-0c2219b4f0ff None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:27:39.247099 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 301/586] 10.222.0.57 () {56 vars in 1374 bytes} [Fri Jul 1 16:27:39 2022] GET /identity/v3/role_assignments?user.id=859fe410c2d2450fb803c6244839f27c&scope.project.id=e0788e6725a84e57a4174284c1845755&role.id=e8809b67c9dc471daa1c2bd42c9a2baf => generated 259 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:27:39.249986 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0831f262-c17d-456c-9509-5bcc78121126 None admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:27:39.353563 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=100 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:27:39.403564 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58666e69-c3e7-443a-8e51-0c2219b4f0ff None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:27:39.412935 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0831f262-c17d-456c-9509-5bcc78121126 None admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:27:39.442964 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9647543-ddc1-442a-b3e8-3545ac000438 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:27:39.443702 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9647543-ddc1-442a-b3e8-3545ac000438 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:39.443993 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9647543-ddc1-442a-b3e8-3545ac000438 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:39.444270 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9647543-ddc1-442a-b3e8-3545ac000438 None admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:39.446297 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9647543-ddc1-442a-b3e8-3545ac000438 None admin] RBAC: Adding query filter params (name=service) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:27:39.446642 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9647543-ddc1-442a-b3e8-3545ac000438 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:27:39.448048 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9647543-ddc1-442a-b3e8-3545ac000438 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:27:39.457674 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 286/587] 10.222.0.57 () {56 vars in 1103 bytes} [Fri Jul 1 16:27:39 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:27:39.458779 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9647543-ddc1-442a-b3e8-3545ac000438 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:27:39.462803 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2101ce10-b80e-49aa-af6a-6b384889b841 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:27:39.464929 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2101ce10-b80e-49aa-af6a-6b384889b841 None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:39.465397 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2101ce10-b80e-49aa-af6a-6b384889b841 None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:39.465838 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2101ce10-b80e-49aa-af6a-6b384889b841 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:39.476959 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2101ce10-b80e-49aa-af6a-6b384889b841 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:27:39.479483 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2101ce10-b80e-49aa-af6a-6b384889b841 None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:27:39.498718 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application [None req-2101ce10-b80e-49aa-af6a-6b384889b841 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 01 16:27:39.498718 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 16:27:39.498718 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 16:27:39.498718 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 16:27:39.498718 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 16:27:39.498718 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 16:27:39.498718 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 16:27:39.498718 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 16:27:39.498718 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 16:27:39.498718 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 16:27:39.498718 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 16:27:39.498718 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 16:27:39.498718 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 01 16:27:39.498718 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 01 16:27:39.498718 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 01 16:27:39.498718 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 01 16:27:39.499960 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 16:27:39.499960 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 16:27:39.499960 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 01 16:27:39.499960 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 01 16:27:39.499960 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 01 16:27:39.499960 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 01 16:27:39.499960 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 16:27:39.499960 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 16:27:39.499960 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 16:27:39.499960 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 16:27:39.499960 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 16:27:39.499960 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application with Lock( Jul 01 16:27:39.499960 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 16:27:39.499960 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self._enter() Jul 01 16:27:39.499960 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 16:27:39.499960 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 16:27:39.499960 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 16:27:39.499960 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.creator() Jul 01 16:27:39.501157 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 16:27:39.501157 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application created_value = creator( Jul 01 16:27:39.501157 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 01 16:27:39.501157 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 01 16:27:39.501157 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 01 16:27:39.501157 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 01 16:27:39.501157 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 01 16:27:39.501157 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 01 16:27:39.501157 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 01 16:27:39.501157 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application Jul 01 16:27:39.504701 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 302/588] 10.222.0.57 () {56 vars in 1099 bytes} [Fri Jul 1 16:27:39 2022] GET /identity/v3/users/cinder => generated 84 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 16:27:39.505843 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2101ce10-b80e-49aa-af6a-6b384889b841 None admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:27:39.542844 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:27:39.596026 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9647543-ddc1-442a-b3e8-3545ac000438 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:27:39.633574 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581dfe29-a33a-4b01-9085-05da4d56a4f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:27:39.634261 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581dfe29-a33a-4b01-9085-05da4d56a4f4 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:39.634565 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581dfe29-a33a-4b01-9085-05da4d56a4f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:39.634849 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581dfe29-a33a-4b01-9085-05da4d56a4f4 None admin] PATH_INFO: `/v3/domains` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:39.635600 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-581dfe29-a33a-4b01-9085-05da4d56a4f4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:27:39.635908 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-581dfe29-a33a-4b01-9085-05da4d56a4f4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:27:39.637329 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-581dfe29-a33a-4b01-9085-05da4d56a4f4 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:27:39.648176 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2101ce10-b80e-49aa-af6a-6b384889b841 None admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:27:39.655780 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 287/589] 10.222.0.57 () {56 vars in 1109 bytes} [Fri Jul 1 16:27:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:27:39.655780 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581dfe29-a33a-4b01-9085-05da4d56a4f4 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:27:39.683023 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=100 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:27:39.697281 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ab983a-37cc-4b65-8ff6-67f86c64ec98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:27:39.698469 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ab983a-37cc-4b65-8ff6-67f86c64ec98 None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:39.698918 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ab983a-37cc-4b65-8ff6-67f86c64ec98 None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:39.699447 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ab983a-37cc-4b65-8ff6-67f86c64ec98 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:39.712986 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ab983a-37cc-4b65-8ff6-67f86c64ec98 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:27:39.715103 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ab983a-37cc-4b65-8ff6-67f86c64ec98 None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:27:39.734119 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application [None req-d9ab983a-37cc-4b65-8ff6-67f86c64ec98 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 16:27:39.734119 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 16:27:39.734119 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 16:27:39.734119 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 16:27:39.734119 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 16:27:39.734119 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 16:27:39.734119 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 16:27:39.734119 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 16:27:39.734119 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 16:27:39.734119 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 16:27:39.734119 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 16:27:39.734119 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 16:27:39.734119 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 01 16:27:39.734119 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 01 16:27:39.734119 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 01 16:27:39.735293 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 01 16:27:39.735293 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 16:27:39.735293 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 16:27:39.735293 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 16:27:39.735293 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 16:27:39.735293 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 16:27:39.735293 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 16:27:39.735293 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 16:27:39.735293 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application with Lock( Jul 01 16:27:39.735293 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 16:27:39.735293 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self._enter() Jul 01 16:27:39.735293 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 16:27:39.735293 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 16:27:39.735293 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 16:27:39.735293 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.creator() Jul 01 16:27:39.735293 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 16:27:39.735293 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application created_value = creator( Jul 01 16:27:39.736377 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 01 16:27:39.736377 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 01 16:27:39.736377 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 16:27:39.736377 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application Jul 01 16:27:39.738265 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 303/590] 10.222.0.57 () {56 vars in 1108 bytes} [Fri Jul 1 16:27:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 16:27:39.739305 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ab983a-37cc-4b65-8ff6-67f86c64ec98 None admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:27:39.795650 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581dfe29-a33a-4b01-9085-05da4d56a4f4 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:27:39.831181 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f45541-8ed6-4ae5-a9b2-7f165e60ac79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:27:39.831903 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f45541-8ed6-4ae5-a9b2-7f165e60ac79 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:39.832137 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f45541-8ed6-4ae5-a9b2-7f165e60ac79 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:39.832420 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f45541-8ed6-4ae5-a9b2-7f165e60ac79 None admin] PATH_INFO: `/v3/users` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:39.833272 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f45541-8ed6-4ae5-a9b2-7f165e60ac79 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:27:39.833583 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f45541-8ed6-4ae5-a9b2-7f165e60ac79 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:27:39.835636 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f45541-8ed6-4ae5-a9b2-7f165e60ac79 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:27:39.838705 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:27:39.855593 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 288/591] 10.222.0.57 () {56 vars in 1137 bytes} [Fri Jul 1 16:27:39 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:27:39.856453 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f45541-8ed6-4ae5-a9b2-7f165e60ac79 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:27:39.860324 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ab983a-37cc-4b65-8ff6-67f86c64ec98 None admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:27:39.896742 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af261860-4317-430f-bf57-1c5dc7056d4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:27:39.898405 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af261860-4317-430f-bf57-1c5dc7056d4b None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:39.898855 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af261860-4317-430f-bf57-1c5dc7056d4b None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:39.899286 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af261860-4317-430f-bf57-1c5dc7056d4b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:39.906602 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af261860-4317-430f-bf57-1c5dc7056d4b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:27:39.908271 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af261860-4317-430f-bf57-1c5dc7056d4b None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:27:39.923911 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application [None req-af261860-4317-430f-bf57-1c5dc7056d4b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 01 16:27:39.923911 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 16:27:39.923911 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 16:27:39.923911 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 16:27:39.923911 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 16:27:39.923911 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 16:27:39.923911 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 16:27:39.923911 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 16:27:39.923911 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 16:27:39.923911 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 16:27:39.923911 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 16:27:39.923911 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 16:27:39.923911 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 01 16:27:39.923911 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 01 16:27:39.923911 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 01 16:27:39.925070 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 01 16:27:39.925070 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 16:27:39.925070 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 16:27:39.925070 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 16:27:39.925070 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 16:27:39.925070 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 16:27:39.925070 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 16:27:39.925070 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 16:27:39.925070 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application with Lock( Jul 01 16:27:39.925070 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 16:27:39.925070 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self._enter() Jul 01 16:27:39.925070 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 16:27:39.925070 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 16:27:39.925070 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 16:27:39.925070 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.creator() Jul 01 16:27:39.925070 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 16:27:39.925070 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application created_value = creator( Jul 01 16:27:39.926087 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 01 16:27:39.926087 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 01 16:27:39.926087 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 01 16:27:39.926087 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 01 16:27:39.926087 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 01 16:27:39.926087 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 01 16:27:39.926087 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 01 16:27:39.926087 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application Jul 01 16:27:39.933461 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 304/592] 10.222.0.57 () {56 vars in 1111 bytes} [Fri Jul 1 16:27:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 16:27:39.934548 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af261860-4317-430f-bf57-1c5dc7056d4b None admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:27:39.963528 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f45541-8ed6-4ae5-a9b2-7f165e60ac79 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:27:39.972123 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=100 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:27:39.999583 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5ba4cf-4adf-492f-9925-08dc16c4f93f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:27:40.000250 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5ba4cf-4adf-492f-9925-08dc16c4f93f None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:40.000537 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5ba4cf-4adf-492f-9925-08dc16c4f93f None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:40.000824 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5ba4cf-4adf-492f-9925-08dc16c4f93f None admin] PATH_INFO: `/v3/domains` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:40.001532 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec5ba4cf-4adf-492f-9925-08dc16c4f93f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:27:40.001840 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec5ba4cf-4adf-492f-9925-08dc16c4f93f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:27:40.003258 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec5ba4cf-4adf-492f-9925-08dc16c4f93f None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:27:40.018464 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 289/593] 10.222.0.57 () {56 vars in 1109 bytes} [Fri Jul 1 16:27:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:27:40.019343 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5ba4cf-4adf-492f-9925-08dc16c4f93f None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:27:40.045567 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af261860-4317-430f-bf57-1c5dc7056d4b None admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:27:40.058144 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:27:40.083594 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df042b0-5fec-47b9-972c-4aefdfa6eb5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:27:40.084777 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df042b0-5fec-47b9-972c-4aefdfa6eb5d None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:40.085343 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df042b0-5fec-47b9-972c-4aefdfa6eb5d None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:40.085778 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df042b0-5fec-47b9-972c-4aefdfa6eb5d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:40.096591 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df042b0-5fec-47b9-972c-4aefdfa6eb5d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:27:40.098275 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df042b0-5fec-47b9-972c-4aefdfa6eb5d None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:27:40.120066 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application [None req-2df042b0-5fec-47b9-972c-4aefdfa6eb5d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 16:27:40.120066 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 16:27:40.120066 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 16:27:40.120066 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 16:27:40.120066 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 16:27:40.120066 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 16:27:40.120066 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 16:27:40.120066 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 16:27:40.120066 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 16:27:40.120066 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 16:27:40.120066 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 16:27:40.120066 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 16:27:40.120066 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 01 16:27:40.120066 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 01 16:27:40.120066 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 01 16:27:40.121207 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 01 16:27:40.121207 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 16:27:40.121207 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 16:27:40.121207 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 16:27:40.121207 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 16:27:40.121207 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 16:27:40.121207 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 16:27:40.121207 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 16:27:40.121207 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application with Lock( Jul 01 16:27:40.121207 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 16:27:40.121207 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self._enter() Jul 01 16:27:40.121207 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 16:27:40.121207 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 16:27:40.121207 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 16:27:40.121207 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.creator() Jul 01 16:27:40.121207 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 16:27:40.121207 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application created_value = creator( Jul 01 16:27:40.122203 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 01 16:27:40.122203 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 01 16:27:40.122203 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 16:27:40.122203 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application Jul 01 16:27:40.124056 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 305/594] 10.222.0.57 () {56 vars in 1108 bytes} [Fri Jul 1 16:27:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 16:27:40.125010 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df042b0-5fec-47b9-972c-4aefdfa6eb5d None admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:27:40.129463 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5ba4cf-4adf-492f-9925-08dc16c4f93f None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:27:40.165706 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b673552d-2f15-4dfa-93db-2f7dd1ed4b2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:27:40.166391 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b673552d-2f15-4dfa-93db-2f7dd1ed4b2e None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:40.166685 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b673552d-2f15-4dfa-93db-2f7dd1ed4b2e None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:40.166964 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b673552d-2f15-4dfa-93db-2f7dd1ed4b2e None admin] PATH_INFO: `/v3/projects` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:40.167758 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b673552d-2f15-4dfa-93db-2f7dd1ed4b2e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:27:40.168070 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b673552d-2f15-4dfa-93db-2f7dd1ed4b2e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:27:40.169593 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b673552d-2f15-4dfa-93db-2f7dd1ed4b2e None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:27:40.186531 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 290/595] 10.222.0.57 () {56 vars in 1148 bytes} [Fri Jul 1 16:27:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:27:40.186531 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b673552d-2f15-4dfa-93db-2f7dd1ed4b2e None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:27:40.225229 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=100 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:27:40.243795 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df042b0-5fec-47b9-972c-4aefdfa6eb5d None admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:27:40.280400 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64165e32-5be3-4116-8c0b-8075549ff219 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:27:40.281449 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64165e32-5be3-4116-8c0b-8075549ff219 None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:40.281923 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64165e32-5be3-4116-8c0b-8075549ff219 None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:40.282377 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64165e32-5be3-4116-8c0b-8075549ff219 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:40.287306 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:27:40.293464 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64165e32-5be3-4116-8c0b-8075549ff219 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:27:40.295148 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64165e32-5be3-4116-8c0b-8075549ff219 None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:27:40.295611 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b673552d-2f15-4dfa-93db-2f7dd1ed4b2e None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:27:40.311617 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application [None req-64165e32-5be3-4116-8c0b-8075549ff219 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 01 16:27:40.311617 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 16:27:40.311617 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 16:27:40.311617 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 16:27:40.311617 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 16:27:40.311617 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 16:27:40.311617 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 16:27:40.311617 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 16:27:40.311617 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 16:27:40.311617 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 16:27:40.311617 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 16:27:40.311617 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 16:27:40.311617 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 01 16:27:40.311617 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 01 16:27:40.311617 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 01 16:27:40.311617 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 01 16:27:40.312780 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 16:27:40.312780 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 16:27:40.312780 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 01 16:27:40.312780 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 01 16:27:40.312780 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 01 16:27:40.312780 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 01 16:27:40.312780 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 16:27:40.312780 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 16:27:40.312780 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 16:27:40.312780 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 16:27:40.312780 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 16:27:40.312780 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application with Lock( Jul 01 16:27:40.312780 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 16:27:40.312780 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self._enter() Jul 01 16:27:40.312780 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 16:27:40.312780 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 16:27:40.312780 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 16:27:40.312780 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.creator() Jul 01 16:27:40.313864 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 16:27:40.313864 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application created_value = creator( Jul 01 16:27:40.313864 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 01 16:27:40.313864 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 01 16:27:40.313864 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 01 16:27:40.313864 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 01 16:27:40.313864 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 01 16:27:40.313864 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 01 16:27:40.313864 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 01 16:27:40.313864 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application Jul 01 16:27:40.316019 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 306/596] 10.222.0.57 () {56 vars in 1102 bytes} [Fri Jul 1 16:27:40 2022] GET /identity/v3/users/neutron => generated 85 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 16:27:40.316925 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64165e32-5be3-4116-8c0b-8075549ff219 None admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:27:40.338681 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e3471f-53f9-484c-a47a-bab8e154cb87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:27:40.339354 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e3471f-53f9-484c-a47a-bab8e154cb87 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:40.339641 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e3471f-53f9-484c-a47a-bab8e154cb87 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:40.339918 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e3471f-53f9-484c-a47a-bab8e154cb87 None admin] PATH_INFO: `/v3/domains` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:40.340649 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e3471f-53f9-484c-a47a-bab8e154cb87 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:27:40.340957 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e3471f-53f9-484c-a47a-bab8e154cb87 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:27:40.342470 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e3471f-53f9-484c-a47a-bab8e154cb87 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:27:40.356370 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 291/597] 10.222.0.57 () {56 vars in 1109 bytes} [Fri Jul 1 16:27:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:27:40.357284 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e3471f-53f9-484c-a47a-bab8e154cb87 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:27:40.447441 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64165e32-5be3-4116-8c0b-8075549ff219 None admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:27:40.482114 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e3471f-53f9-484c-a47a-bab8e154cb87 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:27:40.493547 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8a7a04-d887-4e40-bb75-1a4342b0f344 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:27:40.494687 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8a7a04-d887-4e40-bb75-1a4342b0f344 None admin] REQUEST_METHOD: `PUT` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:40.495152 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8a7a04-d887-4e40-bb75-1a4342b0f344 None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:40.495675 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8a7a04-d887-4e40-bb75-1a4342b0f344 None admin] PATH_INFO: `/v3/projects/e0788e6725a84e57a4174284c1845755/users/fd028715bfea43099e7c6cde429ecf92/roles/e8809b67c9dc471daa1c2bd42c9a2baf` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:40.514167 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f8a7a04-d887-4e40-bb75-1a4342b0f344 None admin] RBAC: Authorizing `identity:create_grant(project_id=e0788e6725a84e57a4174284c1845755, user_id=fd028715bfea43099e7c6cde429ecf92, role_id=e8809b67c9dc471daa1c2bd42c9a2baf)` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:27:40.516218 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f8a7a04-d887-4e40-bb75-1a4342b0f344 None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:27:40.517675 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c76f2f-3fbc-401a-a27d-63878d9707f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:27:40.518373 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c76f2f-3fbc-401a-a27d-63878d9707f6 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:40.518660 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c76f2f-3fbc-401a-a27d-63878d9707f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:40.518940 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c76f2f-3fbc-401a-a27d-63878d9707f6 None admin] PATH_INFO: `/v3/users` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:40.519789 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c76f2f-3fbc-401a-a27d-63878d9707f6 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:27:40.520098 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c76f2f-3fbc-401a-a27d-63878d9707f6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:27:40.521577 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c76f2f-3fbc-401a-a27d-63878d9707f6 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:27:40.521736 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:27:40.536682 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 307/598] 10.222.0.57 () {58 vars in 1439 bytes} [Fri Jul 1 16:27:40 2022] PUT /identity/v3/projects/e0788e6725a84e57a4174284c1845755/users/fd028715bfea43099e7c6cde429ecf92/roles/e8809b67c9dc471daa1c2bd42c9a2baf => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 16:27:40.539089 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8a7a04-d887-4e40-bb75-1a4342b0f344 None admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:27:40.550626 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 292/599] 10.222.0.57 () {56 vars in 1139 bytes} [Fri Jul 1 16:27:40 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:27:40.550626 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c76f2f-3fbc-401a-a27d-63878d9707f6 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:27:40.564995 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:27:40.669792 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8a7a04-d887-4e40-bb75-1a4342b0f344 None admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:27:40.709625 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c76f2f-3fbc-401a-a27d-63878d9707f6 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:27:40.712939 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84ea73c-dbe5-4531-9d49-880aa788a8f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:27:40.714435 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84ea73c-dbe5-4531-9d49-880aa788a8f0 None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:40.714920 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84ea73c-dbe5-4531-9d49-880aa788a8f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:40.715379 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84ea73c-dbe5-4531-9d49-880aa788a8f0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:40.729953 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b84ea73c-dbe5-4531-9d49-880aa788a8f0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:27:40.731884 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b84ea73c-dbe5-4531-9d49-880aa788a8f0 None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:27:40.756310 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ca0072-8499-4535-b39f-caf2ff3fd4ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:27:40.758176 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ca0072-8499-4535-b39f-caf2ff3fd4ac None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:40.758295 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application [None req-b84ea73c-dbe5-4531-9d49-880aa788a8f0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 01 16:27:40.758295 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 16:27:40.758295 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 16:27:40.758295 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 16:27:40.758295 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 16:27:40.758295 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 16:27:40.758295 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 16:27:40.758295 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 16:27:40.758295 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 16:27:40.758295 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 16:27:40.758295 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 16:27:40.758295 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 16:27:40.758295 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 01 16:27:40.758295 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 01 16:27:40.758295 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 01 16:27:40.759212 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 01 16:27:40.759212 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 16:27:40.759212 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 16:27:40.759212 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 16:27:40.759212 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 16:27:40.759212 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 16:27:40.759212 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 16:27:40.759212 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 16:27:40.759212 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application with Lock( Jul 01 16:27:40.759212 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 16:27:40.759212 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self._enter() Jul 01 16:27:40.759212 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 16:27:40.759212 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 16:27:40.759212 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 16:27:40.759212 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.creator() Jul 01 16:27:40.759212 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 16:27:40.759212 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application created_value = creator( Jul 01 16:27:40.760120 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 01 16:27:40.760120 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 01 16:27:40.760120 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 01 16:27:40.760120 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 01 16:27:40.760120 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 01 16:27:40.760120 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 01 16:27:40.760120 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 01 16:27:40.760120 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application Jul 01 16:27:40.760120 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 308/600] 10.222.0.57 () {56 vars in 1111 bytes} [Fri Jul 1 16:27:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 16:27:40.761470 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ca0072-8499-4535-b39f-caf2ff3fd4ac None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:40.761932 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b56afb-c372-42d3-aabf-282fa802aff3 None None] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:40.762286 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b56afb-c372-42d3-aabf-282fa802aff3 None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:40.762607 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ca0072-8499-4535-b39f-caf2ff3fd4ac None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:40.762709 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b56afb-c372-42d3-aabf-282fa802aff3 None None] PATH_INFO: `/` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:40.769320 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 309/601] 10.222.0.57 () {54 vars in 926 bytes} [Fri Jul 1 16:27:40 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 16:27:40.770766 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b56afb-c372-42d3-aabf-282fa802aff3 None None] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:27:40.783976 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ca0072-8499-4535-b39f-caf2ff3fd4ac None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:27:40.785764 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ca0072-8499-4535-b39f-caf2ff3fd4ac None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:27:40.808823 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application [None req-c4ca0072-8499-4535-b39f-caf2ff3fd4ac None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 16:27:40.808823 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 16:27:40.808823 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 16:27:40.808823 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 16:27:40.808823 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 16:27:40.808823 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 16:27:40.808823 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 16:27:40.808823 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 16:27:40.808823 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 16:27:40.808823 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 16:27:40.808823 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 16:27:40.808823 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 16:27:40.808823 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 01 16:27:40.808823 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 01 16:27:40.808823 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 01 16:27:40.809757 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 01 16:27:40.809757 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 16:27:40.809757 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 16:27:40.809757 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 16:27:40.809757 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 16:27:40.809757 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 16:27:40.809757 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 16:27:40.809757 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 16:27:40.809757 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application with Lock( Jul 01 16:27:40.809757 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 16:27:40.809757 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._enter() Jul 01 16:27:40.809757 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 16:27:40.809757 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 16:27:40.809757 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 16:27:40.809757 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.creator() Jul 01 16:27:40.809757 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 16:27:40.809757 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application created_value = creator( Jul 01 16:27:40.810693 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 01 16:27:40.810693 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 01 16:27:40.810693 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 16:27:40.810693 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Jul 01 16:27:40.812706 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 293/602] 10.222.0.57 () {56 vars in 1108 bytes} [Fri Jul 1 16:27:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 265 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 01 16:27:40.814107 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e4116f-120d-48f4-a356-06e4dd57e891 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:40.814598 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e4116f-120d-48f4-a356-06e4dd57e891 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:40.815072 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e4116f-120d-48f4-a356-06e4dd57e891 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:40.832979 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-d7e4116f-120d-48f4-a356-06e4dd57e891 None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:27:40.834532 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-d7e4116f-120d-48f4-a356-06e4dd57e891 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:27:40.846687 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=100 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:27:40.913870 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:27:40.916906 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b56afb-c372-42d3-aabf-282fa802aff3 None None] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:27:40.930894 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 294/603] 10.222.0.57 () {58 vars in 1023 bytes} [Fri Jul 1 16:27:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 01 16:27:40.930894 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e4116f-120d-48f4-a356-06e4dd57e891 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:27:40.982681 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b59c0ec-8766-42ca-89fc-07f7f7e82b28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:27:40.984372 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b59c0ec-8766-42ca-89fc-07f7f7e82b28 None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:40.985932 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b59c0ec-8766-42ca-89fc-07f7f7e82b28 None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:40.986494 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b59c0ec-8766-42ca-89fc-07f7f7e82b28 None admin] PATH_INFO: `/v3/projects` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:40.987531 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b59c0ec-8766-42ca-89fc-07f7f7e82b28 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80312) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:27:40.988053 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b59c0ec-8766-42ca-89fc-07f7f7e82b28 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:27:40.989719 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b59c0ec-8766-42ca-89fc-07f7f7e82b28 None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:27:41.006599 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 310/604] 10.222.0.57 () {56 vars in 1148 bytes} [Fri Jul 1 16:27:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:27:41.008486 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b59c0ec-8766-42ca-89fc-07f7f7e82b28 None admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:27:41.081491 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e4116f-120d-48f4-a356-06e4dd57e891 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:27:41.124852 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf93ac5-8aa6-4d34-be53-097d0e93746c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:27:41.126140 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf93ac5-8aa6-4d34-be53-097d0e93746c None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:41.126679 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf93ac5-8aa6-4d34-be53-097d0e93746c None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:41.127582 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf93ac5-8aa6-4d34-be53-097d0e93746c None admin] PATH_INFO: `/v3/domains` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:41.128532 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf93ac5-8aa6-4d34-be53-097d0e93746c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:27:41.129313 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf93ac5-8aa6-4d34-be53-097d0e93746c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:27:41.130927 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf93ac5-8aa6-4d34-be53-097d0e93746c None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:27:41.153899 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b59c0ec-8766-42ca-89fc-07f7f7e82b28 None admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:27:41.154377 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 295/605] 10.222.0.57 () {56 vars in 1109 bytes} [Fri Jul 1 16:27:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:27:41.156330 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf93ac5-8aa6-4d34-be53-097d0e93746c None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:27:41.198599 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32354e4c-732f-473c-b8a7-23f57258d808 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:27:41.199636 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32354e4c-732f-473c-b8a7-23f57258d808 None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:41.200021 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32354e4c-732f-473c-b8a7-23f57258d808 None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:41.200404 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32354e4c-732f-473c-b8a7-23f57258d808 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:41.202680 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=100 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:27:41.213435 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32354e4c-732f-473c-b8a7-23f57258d808 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 01 16:27:41.221542 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application [None req-32354e4c-732f-473c-b8a7-23f57258d808 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 01 16:27:41.221542 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 16:27:41.221542 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 16:27:41.221542 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 16:27:41.221542 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 16:27:41.221542 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 16:27:41.221542 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 16:27:41.221542 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 16:27:41.221542 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 16:27:41.221542 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 16:27:41.221542 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 16:27:41.221542 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 16:27:41.221542 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 01 16:27:41.221542 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 01 16:27:41.221542 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 01 16:27:41.221542 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 01 16:27:41.222709 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 01 16:27:41.222709 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 01 16:27:41.222709 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 01 16:27:41.222709 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 01 16:27:41.222709 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 16:27:41.222709 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 16:27:41.222709 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 16:27:41.222709 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 16:27:41.222709 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 16:27:41.222709 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 16:27:41.222709 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 16:27:41.222709 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application with Lock( Jul 01 16:27:41.222709 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 16:27:41.222709 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self._enter() Jul 01 16:27:41.222709 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 16:27:41.222709 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 16:27:41.223738 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 16:27:41.223738 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.creator() Jul 01 16:27:41.223738 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 16:27:41.223738 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application created_value = creator( Jul 01 16:27:41.223738 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 01 16:27:41.223738 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 01 16:27:41.223738 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 01 16:27:41.223738 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 01 16:27:41.223738 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 01 16:27:41.223738 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 01 16:27:41.223738 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 01 16:27:41.223738 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application Jul 01 16:27:41.226381 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 311/606] 10.222.0.57 () {56 vars in 1102 bytes} [Fri Jul 1 16:27:41 2022] GET /identity/v3/roles/service => generated 85 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 16:27:41.226898 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32354e4c-732f-473c-b8a7-23f57258d808 None admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:27:41.237141 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:27:41.289837 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf93ac5-8aa6-4d34-be53-097d0e93746c None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:27:41.328570 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9baed4db-962f-4404-8be2-b0b4848c9836 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:27:41.329889 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9baed4db-962f-4404-8be2-b0b4848c9836 None admin] REQUEST_METHOD: `PUT` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:41.330271 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9baed4db-962f-4404-8be2-b0b4848c9836 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:41.330676 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9baed4db-962f-4404-8be2-b0b4848c9836 None admin] PATH_INFO: `/v3/projects/e0788e6725a84e57a4174284c1845755/users/dbddf971a2354bbdbb9da1d63f3abf88/roles/e8809b67c9dc471daa1c2bd42c9a2baf` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:41.352533 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9baed4db-962f-4404-8be2-b0b4848c9836 None admin] RBAC: Authorizing `identity:create_grant(project_id=e0788e6725a84e57a4174284c1845755, user_id=dbddf971a2354bbdbb9da1d63f3abf88, role_id=e8809b67c9dc471daa1c2bd42c9a2baf)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:27:41.353994 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9baed4db-962f-4404-8be2-b0b4848c9836 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:27:41.364661 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32354e4c-732f-473c-b8a7-23f57258d808 None admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:27:41.378259 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 296/607] 10.222.0.57 () {58 vars in 1439 bytes} [Fri Jul 1 16:27:41 2022] PUT /identity/v3/projects/e0788e6725a84e57a4174284c1845755/users/dbddf971a2354bbdbb9da1d63f3abf88/roles/e8809b67c9dc471daa1c2bd42c9a2baf => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 16:27:41.380061 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9baed4db-962f-4404-8be2-b0b4848c9836 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:27:41.416373 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19bab16d-01cc-4bfd-aa1f-476b821b098a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:27:41.420551 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bab16d-01cc-4bfd-aa1f-476b821b098a None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:41.422316 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bab16d-01cc-4bfd-aa1f-476b821b098a None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:41.425883 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bab16d-01cc-4bfd-aa1f-476b821b098a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:41.446552 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19bab16d-01cc-4bfd-aa1f-476b821b098a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:27:41.449339 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19bab16d-01cc-4bfd-aa1f-476b821b098a None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:27:41.474735 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application [None req-19bab16d-01cc-4bfd-aa1f-476b821b098a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 01 16:27:41.474735 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 16:27:41.474735 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 16:27:41.474735 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 16:27:41.474735 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 16:27:41.474735 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 16:27:41.474735 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 16:27:41.474735 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 16:27:41.474735 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 16:27:41.474735 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 16:27:41.474735 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 16:27:41.474735 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 16:27:41.474735 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 01 16:27:41.474735 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 01 16:27:41.474735 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 01 16:27:41.475839 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 01 16:27:41.475839 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 16:27:41.475839 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 16:27:41.475839 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 16:27:41.475839 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 16:27:41.475839 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 16:27:41.475839 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 16:27:41.475839 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 16:27:41.475839 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application with Lock( Jul 01 16:27:41.475839 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 16:27:41.475839 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self._enter() Jul 01 16:27:41.475839 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 16:27:41.475839 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 16:27:41.475839 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 16:27:41.475839 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.creator() Jul 01 16:27:41.475839 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 16:27:41.475839 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application created_value = creator( Jul 01 16:27:41.477205 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 01 16:27:41.477205 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 01 16:27:41.477205 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 01 16:27:41.477205 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 01 16:27:41.477205 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 01 16:27:41.477205 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 01 16:27:41.477205 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 01 16:27:41.477205 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application Jul 01 16:27:41.484952 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 312/608] 10.222.0.57 () {56 vars in 1111 bytes} [Fri Jul 1 16:27:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 16:27:41.487005 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86470a5-cfa5-472f-8246-6b00b06173c1 None None] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:41.487518 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86470a5-cfa5-472f-8246-6b00b06173c1 None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:41.488055 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86470a5-cfa5-472f-8246-6b00b06173c1 None None] PATH_INFO: `/` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:41.490337 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 313/609] 10.222.0.57 () {54 vars in 926 bytes} [Fri Jul 1 16:27:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 16:27:41.492268 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86470a5-cfa5-472f-8246-6b00b06173c1 None None] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:27:41.544291 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=100 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:27:41.555774 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9baed4db-962f-4404-8be2-b0b4848c9836 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:27:41.566510 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:27:41.607690 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec30be0a-b04c-4d33-aa71-5bac7d2530ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:27:41.608661 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec30be0a-b04c-4d33-aa71-5bac7d2530ae None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:41.608948 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec30be0a-b04c-4d33-aa71-5bac7d2530ae None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:41.609734 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec30be0a-b04c-4d33-aa71-5bac7d2530ae None admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:41.610901 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec30be0a-b04c-4d33-aa71-5bac7d2530ae None admin] RBAC: Adding query filter params (name=service) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:27:41.611218 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec30be0a-b04c-4d33-aa71-5bac7d2530ae None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:27:41.613144 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec30be0a-b04c-4d33-aa71-5bac7d2530ae None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:27:41.623251 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 297/610] 10.222.0.57 () {56 vars in 1103 bytes} [Fri Jul 1 16:27:41 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:27:41.624622 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b187c4e6-9f5b-40ca-bf04-eecd40761e05 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:41.624917 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b187c4e6-9f5b-40ca-bf04-eecd40761e05 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:41.625263 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b187c4e6-9f5b-40ca-bf04-eecd40761e05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:41.628414 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86470a5-cfa5-472f-8246-6b00b06173c1 None None] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:27:41.642883 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-b187c4e6-9f5b-40ca-bf04-eecd40761e05 None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:27:41.644067 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-b187c4e6-9f5b-40ca-bf04-eecd40761e05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:27:41.667005 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caff7e3c-46ce-4bf8-891e-6a0f5b4a0247 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:27:41.667725 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caff7e3c-46ce-4bf8-891e-6a0f5b4a0247 None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:41.668013 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caff7e3c-46ce-4bf8-891e-6a0f5b4a0247 None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:41.668292 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caff7e3c-46ce-4bf8-891e-6a0f5b4a0247 None admin] PATH_INFO: `/v3/projects` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:41.669085 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caff7e3c-46ce-4bf8-891e-6a0f5b4a0247 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80312) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:27:41.669396 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caff7e3c-46ce-4bf8-891e-6a0f5b4a0247 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:27:41.670867 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caff7e3c-46ce-4bf8-891e-6a0f5b4a0247 None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:27:41.686154 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 314/611] 10.222.0.57 () {56 vars in 1148 bytes} [Fri Jul 1 16:27:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:27:41.687024 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caff7e3c-46ce-4bf8-891e-6a0f5b4a0247 None admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:27:41.731111 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 298/612] 10.222.0.57 () {58 vars in 1023 bytes} [Fri Jul 1 16:27:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 01 16:27:41.731725 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b187c4e6-9f5b-40ca-bf04-eecd40761e05 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:27:41.806686 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=100 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:27:41.814262 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caff7e3c-46ce-4bf8-891e-6a0f5b4a0247 None admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:27:41.856587 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda3968b-e07d-45df-91ea-7c797e423f5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:27:41.857460 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda3968b-e07d-45df-91ea-7c797e423f5f None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:41.857812 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda3968b-e07d-45df-91ea-7c797e423f5f None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:41.858156 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda3968b-e07d-45df-91ea-7c797e423f5f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:41.869153 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:27:41.870301 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda3968b-e07d-45df-91ea-7c797e423f5f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:27:41.872213 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda3968b-e07d-45df-91ea-7c797e423f5f None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:27:41.876061 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b187c4e6-9f5b-40ca-bf04-eecd40761e05 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:27:41.891024 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application [None req-bda3968b-e07d-45df-91ea-7c797e423f5f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 16:27:41.891024 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 16:27:41.891024 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 16:27:41.891024 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 16:27:41.891024 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 16:27:41.891024 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 16:27:41.891024 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 16:27:41.891024 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 16:27:41.891024 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 16:27:41.891024 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 16:27:41.891024 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 16:27:41.891024 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 16:27:41.891024 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 01 16:27:41.891024 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 01 16:27:41.891024 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 01 16:27:41.892195 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 01 16:27:41.892195 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 16:27:41.892195 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 16:27:41.892195 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 16:27:41.892195 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 16:27:41.892195 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 16:27:41.892195 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 16:27:41.892195 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 16:27:41.892195 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application with Lock( Jul 01 16:27:41.892195 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 16:27:41.892195 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self._enter() Jul 01 16:27:41.892195 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 16:27:41.892195 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 16:27:41.892195 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 16:27:41.892195 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.creator() Jul 01 16:27:41.892195 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 16:27:41.892195 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application created_value = creator( Jul 01 16:27:41.893303 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 01 16:27:41.893303 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 01 16:27:41.893303 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 16:27:41.893303 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application Jul 01 16:27:41.894299 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 315/613] 10.222.0.57 () {56 vars in 1108 bytes} [Fri Jul 1 16:27:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 16:27:41.895160 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda3968b-e07d-45df-91ea-7c797e423f5f None admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:27:41.917962 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ee8849-bba5-442a-adff-0d9411f2c511 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:27:41.919590 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ee8849-bba5-442a-adff-0d9411f2c511 None admin] REQUEST_METHOD: `PUT` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:41.920052 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ee8849-bba5-442a-adff-0d9411f2c511 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:41.920635 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ee8849-bba5-442a-adff-0d9411f2c511 None admin] PATH_INFO: `/v3/projects/e0788e6725a84e57a4174284c1845755/users/4ed200ce29684c5f80354fa9a8638fe2/roles/e8809b67c9dc471daa1c2bd42c9a2baf` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:41.941404 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ee8849-bba5-442a-adff-0d9411f2c511 None admin] RBAC: Authorizing `identity:create_grant(project_id=e0788e6725a84e57a4174284c1845755, user_id=4ed200ce29684c5f80354fa9a8638fe2, role_id=e8809b67c9dc471daa1c2bd42c9a2baf)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:27:41.943819 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ee8849-bba5-442a-adff-0d9411f2c511 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:27:41.963045 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 299/614] 10.222.0.57 () {58 vars in 1439 bytes} [Fri Jul 1 16:27:41 2022] PUT /identity/v3/projects/e0788e6725a84e57a4174284c1845755/users/4ed200ce29684c5f80354fa9a8638fe2/roles/e8809b67c9dc471daa1c2bd42c9a2baf => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 16:27:41.964186 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ee8849-bba5-442a-adff-0d9411f2c511 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:27:42.049329 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda3968b-e07d-45df-91ea-7c797e423f5f None admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:27:42.090262 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ee8849-bba5-442a-adff-0d9411f2c511 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:27:42.124080 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676efd58-a906-4643-8f15-ed4796943655 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:27:42.124908 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676efd58-a906-4643-8f15-ed4796943655 None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:42.125197 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676efd58-a906-4643-8f15-ed4796943655 None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:42.125477 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676efd58-a906-4643-8f15-ed4796943655 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:42.131438 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4fc78d-b1b0-4eee-8a0e-5f5daa378a06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:27:42.132419 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4fc78d-b1b0-4eee-8a0e-5f5daa378a06 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:42.132853 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4fc78d-b1b0-4eee-8a0e-5f5daa378a06 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:42.133296 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4fc78d-b1b0-4eee-8a0e-5f5daa378a06 None admin] PATH_INFO: `/v3/domains` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:42.134192 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce4fc78d-b1b0-4eee-8a0e-5f5daa378a06 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:27:42.134718 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce4fc78d-b1b0-4eee-8a0e-5f5daa378a06 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:27:42.136271 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce4fc78d-b1b0-4eee-8a0e-5f5daa378a06 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:27:42.145433 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-676efd58-a906-4643-8f15-ed4796943655 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 01 16:27:42.150708 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 300/615] 10.222.0.57 () {56 vars in 1109 bytes} [Fri Jul 1 16:27:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:27:42.156974 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4fc78d-b1b0-4eee-8a0e-5f5daa378a06 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:27:42.159721 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application [None req-676efd58-a906-4643-8f15-ed4796943655 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 01 16:27:42.159721 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 16:27:42.159721 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 16:27:42.159721 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 16:27:42.159721 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 16:27:42.159721 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 16:27:42.159721 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 16:27:42.159721 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 16:27:42.159721 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 16:27:42.159721 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 16:27:42.159721 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 16:27:42.159721 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 16:27:42.159721 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 01 16:27:42.159721 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 01 16:27:42.159721 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 01 16:27:42.159721 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 01 16:27:42.160681 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 01 16:27:42.160681 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 01 16:27:42.160681 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 01 16:27:42.160681 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 01 16:27:42.160681 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 16:27:42.160681 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 16:27:42.160681 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 16:27:42.160681 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 16:27:42.160681 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 16:27:42.160681 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 16:27:42.160681 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 16:27:42.160681 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application with Lock( Jul 01 16:27:42.160681 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 16:27:42.160681 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self._enter() Jul 01 16:27:42.160681 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 16:27:42.160681 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 16:27:42.161509 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 16:27:42.161509 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.creator() Jul 01 16:27:42.161509 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 16:27:42.161509 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application created_value = creator( Jul 01 16:27:42.161509 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 01 16:27:42.161509 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 01 16:27:42.161509 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 01 16:27:42.161509 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 01 16:27:42.161509 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 01 16:27:42.161509 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 01 16:27:42.161509 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 01 16:27:42.161509 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application Jul 01 16:27:42.161509 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 316/616] 10.222.0.57 () {56 vars in 1102 bytes} [Fri Jul 1 16:27:41 2022] GET /identity/v3/roles/service => generated 85 bytes in 266 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 16:27:42.169110 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676efd58-a906-4643-8f15-ed4796943655 None admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:27:42.176129 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:27:42.187641 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=100 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:27:42.302534 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4fc78d-b1b0-4eee-8a0e-5f5daa378a06 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:27:42.340622 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676efd58-a906-4643-8f15-ed4796943655 None admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:27:42.367815 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4c5973-07f1-452c-ad6b-572680af7bb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:27:42.369170 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4c5973-07f1-452c-ad6b-572680af7bb4 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:42.369663 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4c5973-07f1-452c-ad6b-572680af7bb4 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:42.370144 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4c5973-07f1-452c-ad6b-572680af7bb4 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:42.376649 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9ab38c-2128-427f-9170-f054fc767cb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:27:42.377473 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9ab38c-2128-427f-9170-f054fc767cb9 None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:42.378016 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9ab38c-2128-427f-9170-f054fc767cb9 None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:42.378431 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9ab38c-2128-427f-9170-f054fc767cb9 None admin] PATH_INFO: `/v3/roles` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:42.379307 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d9ab38c-2128-427f-9170-f054fc767cb9 None admin] RBAC: Adding query filter params (name=service) {{(pid=80312) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:27:42.379731 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d9ab38c-2128-427f-9170-f054fc767cb9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:27:42.381223 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d9ab38c-2128-427f-9170-f054fc767cb9 None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:27:42.391977 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 317/617] 10.222.0.57 () {56 vars in 1103 bytes} [Fri Jul 1 16:27:42 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 01 16:27:42.393152 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f4c5973-07f1-452c-ad6b-572680af7bb4 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:27:42.398490 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f4c5973-07f1-452c-ad6b-572680af7bb4 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:27:42.413851 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9ab38c-2128-427f-9170-f054fc767cb9 None admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:27:42.417852 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application [None req-2f4c5973-07f1-452c-ad6b-572680af7bb4 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 01 16:27:42.417852 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 16:27:42.417852 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 16:27:42.417852 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 16:27:42.417852 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 16:27:42.417852 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 16:27:42.417852 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 16:27:42.417852 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 16:27:42.417852 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 16:27:42.417852 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 16:27:42.417852 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 16:27:42.417852 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 16:27:42.417852 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 01 16:27:42.417852 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 01 16:27:42.417852 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 01 16:27:42.417852 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 01 16:27:42.418857 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 16:27:42.418857 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 16:27:42.418857 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 01 16:27:42.418857 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 01 16:27:42.418857 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 01 16:27:42.418857 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 01 16:27:42.418857 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 16:27:42.418857 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 16:27:42.418857 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 16:27:42.418857 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 16:27:42.418857 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 16:27:42.418857 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application with Lock( Jul 01 16:27:42.418857 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 16:27:42.418857 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._enter() Jul 01 16:27:42.418857 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 16:27:42.418857 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 16:27:42.418857 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 16:27:42.418857 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.creator() Jul 01 16:27:42.420142 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 16:27:42.420142 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application created_value = creator( Jul 01 16:27:42.420142 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 01 16:27:42.420142 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 01 16:27:42.420142 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 01 16:27:42.420142 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 01 16:27:42.420142 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 01 16:27:42.420142 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 01 16:27:42.420142 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 01 16:27:42.420142 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Jul 01 16:27:42.420142 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 301/618] 10.222.0.57 () {56 vars in 1093 bytes} [Fri Jul 1 16:27:42 2022] GET /identity/v3/users/nova => generated 82 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 16:27:42.427207 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4c5973-07f1-452c-ad6b-572680af7bb4 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:27:42.553974 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=100 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:27:42.576654 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:27:42.616861 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9ab38c-2128-427f-9170-f054fc767cb9 None admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:27:42.640914 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4c5973-07f1-452c-ad6b-572680af7bb4 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:27:42.673220 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382243c8-d009-463a-8043-cd44747575d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:27:42.674019 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382243c8-d009-463a-8043-cd44747575d0 None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:42.674300 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382243c8-d009-463a-8043-cd44747575d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:42.679223 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382243c8-d009-463a-8043-cd44747575d0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:42.691612 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01dfcee4-62ab-42cc-85a1-9c8e9148f7e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:27:42.692292 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01dfcee4-62ab-42cc-85a1-9c8e9148f7e6 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:42.692574 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01dfcee4-62ab-42cc-85a1-9c8e9148f7e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:42.692849 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01dfcee4-62ab-42cc-85a1-9c8e9148f7e6 None admin] PATH_INFO: `/v3/users` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:42.693693 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01dfcee4-62ab-42cc-85a1-9c8e9148f7e6 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:27:42.693998 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01dfcee4-62ab-42cc-85a1-9c8e9148f7e6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:27:42.696264 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01dfcee4-62ab-42cc-85a1-9c8e9148f7e6 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:27:42.707638 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-382243c8-d009-463a-8043-cd44747575d0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:27:42.709116 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-382243c8-d009-463a-8043-cd44747575d0 None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:27:42.729984 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 302/619] 10.222.0.57 () {56 vars in 1133 bytes} [Fri Jul 1 16:27:42 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 303 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:27:42.738109 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application [None req-382243c8-d009-463a-8043-cd44747575d0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 16:27:42.738109 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 16:27:42.738109 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 16:27:42.738109 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 16:27:42.738109 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 16:27:42.738109 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 16:27:42.738109 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 16:27:42.738109 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 16:27:42.738109 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 16:27:42.738109 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 16:27:42.738109 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 16:27:42.738109 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 16:27:42.738109 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 01 16:27:42.738109 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 01 16:27:42.738109 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 01 16:27:42.740343 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 01 16:27:42.740343 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 16:27:42.740343 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 16:27:42.740343 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 16:27:42.740343 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 16:27:42.740343 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 16:27:42.740343 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 16:27:42.740343 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 16:27:42.740343 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application with Lock( Jul 01 16:27:42.740343 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 16:27:42.740343 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self._enter() Jul 01 16:27:42.740343 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 16:27:42.740343 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 16:27:42.740343 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 16:27:42.740343 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.creator() Jul 01 16:27:42.740343 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 16:27:42.740343 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application created_value = creator( Jul 01 16:27:42.743044 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 01 16:27:42.743044 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 01 16:27:42.743044 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 16:27:42.743044 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application Jul 01 16:27:42.743044 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 318/620] 10.222.0.57 () {56 vars in 1108 bytes} [Fri Jul 1 16:27:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 326 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 01 16:27:42.743692 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01dfcee4-62ab-42cc-85a1-9c8e9148f7e6 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:27:42.750464 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382243c8-d009-463a-8043-cd44747575d0 None admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:27:42.872360 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382243c8-d009-463a-8043-cd44747575d0 None admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:27:42.895847 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=100 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:27:42.908140 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aaeb74d-4528-47b8-bf7b-27fa3493c8d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:27:42.908812 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aaeb74d-4528-47b8-bf7b-27fa3493c8d0 None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:42.909096 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aaeb74d-4528-47b8-bf7b-27fa3493c8d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:42.909376 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aaeb74d-4528-47b8-bf7b-27fa3493c8d0 None admin] PATH_INFO: `/v3/domains` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:42.910274 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aaeb74d-4528-47b8-bf7b-27fa3493c8d0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80312) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:27:42.910616 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aaeb74d-4528-47b8-bf7b-27fa3493c8d0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:27:42.911994 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aaeb74d-4528-47b8-bf7b-27fa3493c8d0 None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:27:42.920091 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01dfcee4-62ab-42cc-85a1-9c8e9148f7e6 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:27:42.924662 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 319/621] 10.222.0.57 () {56 vars in 1109 bytes} [Fri Jul 1 16:27:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:27:42.925960 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c15a08-a9c1-472c-80b1-258f2db7afaa None None] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:42.926248 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c15a08-a9c1-472c-80b1-258f2db7afaa None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:42.926546 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c15a08-a9c1-472c-80b1-258f2db7afaa None None] PATH_INFO: `/` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:42.927999 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 320/622] 10.222.0.57 () {54 vars in 926 bytes} [Fri Jul 1 16:27:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 16:27:42.930112 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c15a08-a9c1-472c-80b1-258f2db7afaa None None] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:27:42.954726 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:27:42.967018 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4fbc1bf-f890-4a7d-b2d3-18ef72441b9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:27:42.968450 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fbc1bf-f890-4a7d-b2d3-18ef72441b9e None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:42.969138 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fbc1bf-f890-4a7d-b2d3-18ef72441b9e None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:42.969602 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fbc1bf-f890-4a7d-b2d3-18ef72441b9e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:43.035496 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c15a08-a9c1-472c-80b1-258f2db7afaa None None] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:27:43.047704 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4fbc1bf-f890-4a7d-b2d3-18ef72441b9e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:27:43.049351 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4fbc1bf-f890-4a7d-b2d3-18ef72441b9e None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:27:43.071147 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583016e5-8b98-43a5-afda-39ae0772d99e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:27:43.072078 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583016e5-8b98-43a5-afda-39ae0772d99e None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:43.072457 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583016e5-8b98-43a5-afda-39ae0772d99e None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:43.072832 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583016e5-8b98-43a5-afda-39ae0772d99e None admin] PATH_INFO: `/v3/users/swift` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:43.080355 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-583016e5-8b98-43a5-afda-39ae0772d99e None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:27:43.081907 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-583016e5-8b98-43a5-afda-39ae0772d99e None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:27:43.096610 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application [None req-583016e5-8b98-43a5-afda-39ae0772d99e None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 01 16:27:43.096610 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 16:27:43.096610 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 16:27:43.096610 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 16:27:43.096610 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 16:27:43.096610 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 16:27:43.096610 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 16:27:43.096610 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 16:27:43.096610 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 16:27:43.096610 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 16:27:43.096610 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 16:27:43.096610 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 16:27:43.096610 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 01 16:27:43.096610 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 01 16:27:43.096610 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 01 16:27:43.096610 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 01 16:27:43.097763 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 16:27:43.097763 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 16:27:43.097763 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 01 16:27:43.097763 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 01 16:27:43.097763 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 01 16:27:43.097763 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 01 16:27:43.097763 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 16:27:43.097763 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 16:27:43.097763 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 16:27:43.097763 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 16:27:43.097763 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 16:27:43.097763 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application with Lock( Jul 01 16:27:43.097763 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 16:27:43.097763 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self._enter() Jul 01 16:27:43.097763 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 16:27:43.097763 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 16:27:43.097763 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 16:27:43.097763 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.creator() Jul 01 16:27:43.098908 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 16:27:43.098908 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application created_value = creator( Jul 01 16:27:43.098908 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 01 16:27:43.098908 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 01 16:27:43.098908 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 01 16:27:43.098908 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 01 16:27:43.098908 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 01 16:27:43.098908 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 01 16:27:43.098908 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 01 16:27:43.098908 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application Jul 01 16:27:43.099578 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 321/623] 10.222.0.57 () {56 vars in 1096 bytes} [Fri Jul 1 16:27:42 2022] GET /identity/v3/users/swift => generated 83 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 01 16:27:43.101722 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b82b2aa-f551-480c-afd3-30e53a4318ad None None] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:43.102069 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b82b2aa-f551-480c-afd3-30e53a4318ad None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:43.102436 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b82b2aa-f551-480c-afd3-30e53a4318ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:43.119748 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.auth.core [None req-5b82b2aa-f551-480c-afd3-30e53a4318ad None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80312) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:27:43.121032 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-5b82b2aa-f551-480c-afd3-30e53a4318ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:27:43.156318 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=100 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:27:43.174465 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 303/624] 10.222.0.57 () {58 vars in 1282 bytes} [Fri Jul 1 16:27:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 431 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 01 16:27:43.175816 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4fbc1bf-f890-4a7d-b2d3-18ef72441b9e None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:27:43.190378 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 322/625] 10.222.0.57 () {58 vars in 1023 bytes} [Fri Jul 1 16:27:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 01 16:27:43.191212 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b82b2aa-f551-480c-afd3-30e53a4318ad None None] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:27:43.207670 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:27:43.293572 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4fbc1bf-f890-4a7d-b2d3-18ef72441b9e None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:27:43.294877 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b82b2aa-f551-480c-afd3-30e53a4318ad None None] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:27:43.333786 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f45d89-a01b-41ef-8477-48417d30953a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:27:43.334650 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f45d89-a01b-41ef-8477-48417d30953a None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:43.335052 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f45d89-a01b-41ef-8477-48417d30953a None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:43.336103 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f45d89-a01b-41ef-8477-48417d30953a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:43.336392 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5476d8-47e9-4452-b7b0-6af7b7f464d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:27:43.337723 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5476d8-47e9-4452-b7b0-6af7b7f464d6 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:43.338164 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5476d8-47e9-4452-b7b0-6af7b7f464d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:43.338730 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5476d8-47e9-4452-b7b0-6af7b7f464d6 None admin] PATH_INFO: `/v3/users` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:43.339854 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c5476d8-47e9-4452-b7b0-6af7b7f464d6 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:27:43.340332 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c5476d8-47e9-4452-b7b0-6af7b7f464d6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:27:43.341931 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c5476d8-47e9-4452-b7b0-6af7b7f464d6 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:27:43.348001 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f45d89-a01b-41ef-8477-48417d30953a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:27:43.349602 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f45d89-a01b-41ef-8477-48417d30953a None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:27:43.362441 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 304/626] 10.222.0.57 () {56 vars in 1135 bytes} [Fri Jul 1 16:27:43 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:27:43.363638 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5476d8-47e9-4452-b7b0-6af7b7f464d6 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:27:43.368480 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application [None req-e8f45d89-a01b-41ef-8477-48417d30953a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 16:27:43.368480 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 16:27:43.368480 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 16:27:43.368480 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 16:27:43.368480 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 16:27:43.368480 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 16:27:43.368480 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 16:27:43.368480 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 16:27:43.368480 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 16:27:43.368480 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 16:27:43.368480 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 16:27:43.368480 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 16:27:43.368480 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 01 16:27:43.368480 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 01 16:27:43.368480 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 01 16:27:43.369571 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 01 16:27:43.369571 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 16:27:43.369571 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 16:27:43.369571 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 16:27:43.369571 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 16:27:43.369571 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 16:27:43.369571 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 16:27:43.369571 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 16:27:43.369571 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application with Lock( Jul 01 16:27:43.369571 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 16:27:43.369571 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self._enter() Jul 01 16:27:43.369571 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 16:27:43.369571 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 16:27:43.369571 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 16:27:43.369571 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.creator() Jul 01 16:27:43.369571 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 16:27:43.369571 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application created_value = creator( Jul 01 16:27:43.370527 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 01 16:27:43.370527 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 01 16:27:43.370527 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 16:27:43.370527 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application Jul 01 16:27:43.375364 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 323/627] 10.222.0.57 () {56 vars in 1108 bytes} [Fri Jul 1 16:27:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 16:27:43.376275 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f45d89-a01b-41ef-8477-48417d30953a None admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:27:43.403083 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=100 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:27:43.471275 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:27:43.483785 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f45d89-a01b-41ef-8477-48417d30953a None admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:27:43.490553 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5476d8-47e9-4452-b7b0-6af7b7f464d6 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:27:43.521241 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff463f3-c593-483b-aa8b-9811c405efc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:27:43.522056 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff463f3-c593-483b-aa8b-9811c405efc3 None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:43.522573 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff463f3-c593-483b-aa8b-9811c405efc3 None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:43.522985 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff463f3-c593-483b-aa8b-9811c405efc3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:43.529702 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f458a742-9f3c-42af-9f7a-59d8099600c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:27:43.530835 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f458a742-9f3c-42af-9f7a-59d8099600c3 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:43.531320 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f458a742-9f3c-42af-9f7a-59d8099600c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:43.531780 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f458a742-9f3c-42af-9f7a-59d8099600c3 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:43.534934 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff463f3-c593-483b-aa8b-9811c405efc3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:27:43.536405 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff463f3-c593-483b-aa8b-9811c405efc3 None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:27:43.543697 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f458a742-9f3c-42af-9f7a-59d8099600c3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 01 16:27:43.551910 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application [None req-f458a742-9f3c-42af-9f7a-59d8099600c3 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 01 16:27:43.551910 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 16:27:43.551910 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 16:27:43.551910 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 16:27:43.551910 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 16:27:43.551910 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 16:27:43.551910 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 16:27:43.551910 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 16:27:43.551910 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 16:27:43.551910 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 16:27:43.551910 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 16:27:43.551910 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 16:27:43.551910 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 01 16:27:43.551910 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 01 16:27:43.551910 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 01 16:27:43.551910 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 01 16:27:43.552918 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 01 16:27:43.552918 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 01 16:27:43.552918 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 01 16:27:43.552918 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 01 16:27:43.552918 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 16:27:43.552918 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 16:27:43.552918 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 16:27:43.552918 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 16:27:43.552918 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 16:27:43.552918 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 16:27:43.552918 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 16:27:43.552918 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application with Lock( Jul 01 16:27:43.552918 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 16:27:43.552918 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._enter() Jul 01 16:27:43.552918 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 16:27:43.552918 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 16:27:43.554276 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 16:27:43.554276 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.creator() Jul 01 16:27:43.554276 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 16:27:43.554276 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application created_value = creator( Jul 01 16:27:43.554276 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 01 16:27:43.554276 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 01 16:27:43.554276 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 01 16:27:43.554276 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 01 16:27:43.554276 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 01 16:27:43.554276 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 01 16:27:43.554276 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 01 16:27:43.554276 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Jul 01 16:27:43.556575 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application [None req-5ff463f3-c593-483b-aa8b-9811c405efc3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 16:27:43.556575 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 16:27:43.556575 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 16:27:43.556575 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 16:27:43.556575 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 16:27:43.556575 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 16:27:43.556575 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 16:27:43.556575 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 16:27:43.556575 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 16:27:43.556575 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 16:27:43.556575 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 16:27:43.556575 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 16:27:43.556575 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 01 16:27:43.556575 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 01 16:27:43.556575 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 01 16:27:43.557412 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 01 16:27:43.557412 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 16:27:43.557412 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 16:27:43.557412 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 16:27:43.557412 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 16:27:43.557412 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 16:27:43.557412 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 16:27:43.557412 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 16:27:43.557412 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application with Lock( Jul 01 16:27:43.557412 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 16:27:43.557412 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self._enter() Jul 01 16:27:43.557412 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 16:27:43.557412 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 16:27:43.557412 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 16:27:43.557412 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.creator() Jul 01 16:27:43.557412 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 16:27:43.557412 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application created_value = creator( Jul 01 16:27:43.558295 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 01 16:27:43.558295 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 01 16:27:43.558295 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 16:27:43.558295 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application Jul 01 16:27:43.558295 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 324/628] 10.222.0.57 () {56 vars in 1108 bytes} [Fri Jul 1 16:27:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 16:27:43.559027 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff463f3-c593-483b-aa8b-9811c405efc3 None admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:27:43.564109 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 305/629] 10.222.0.57 () {56 vars in 1102 bytes} [Fri Jul 1 16:27:43 2022] GET /identity/v3/roles/service => generated 85 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 16:27:43.565197 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f458a742-9f3c-42af-9f7a-59d8099600c3 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:27:43.663590 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=100 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:27:43.684170 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f458a742-9f3c-42af-9f7a-59d8099600c3 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:27:43.693584 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff463f3-c593-483b-aa8b-9811c405efc3 None admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:27:43.724694 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39fa9e6-8438-4876-b7f5-bef93205a932 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:27:43.725687 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39fa9e6-8438-4876-b7f5-bef93205a932 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:43.726264 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39fa9e6-8438-4876-b7f5-bef93205a932 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:43.726752 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39fa9e6-8438-4876-b7f5-bef93205a932 None admin] PATH_INFO: `/v3/domains` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:43.727741 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e39fa9e6-8438-4876-b7f5-bef93205a932 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:27:43.728234 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e39fa9e6-8438-4876-b7f5-bef93205a932 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:27:43.729981 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e39fa9e6-8438-4876-b7f5-bef93205a932 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:27:43.739082 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2235523e-3355-4217-82dc-14923ee80695 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:27:43.739760 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2235523e-3355-4217-82dc-14923ee80695 None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:43.740072 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2235523e-3355-4217-82dc-14923ee80695 None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:43.740884 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2235523e-3355-4217-82dc-14923ee80695 None admin] PATH_INFO: `/v3/domains` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:43.742206 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2235523e-3355-4217-82dc-14923ee80695 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80312) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:27:43.742691 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2235523e-3355-4217-82dc-14923ee80695 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:27:43.744211 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2235523e-3355-4217-82dc-14923ee80695 None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:27:43.745405 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 306/630] 10.222.0.57 () {56 vars in 1109 bytes} [Fri Jul 1 16:27:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:27:43.746694 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39fa9e6-8438-4876-b7f5-bef93205a932 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:27:43.751222 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:27:43.760083 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 325/631] 10.222.0.57 () {56 vars in 1109 bytes} [Fri Jul 1 16:27:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:27:43.762846 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2235523e-3355-4217-82dc-14923ee80695 None admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:27:43.868792 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39fa9e6-8438-4876-b7f5-bef93205a932 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:27:43.890114 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2235523e-3355-4217-82dc-14923ee80695 None admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:27:43.906901 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ba1924-80c9-45a7-8a78-0de4fc1a6d5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:27:43.907569 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ba1924-80c9-45a7-8a78-0de4fc1a6d5a None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:43.907851 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ba1924-80c9-45a7-8a78-0de4fc1a6d5a None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:43.908125 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ba1924-80c9-45a7-8a78-0de4fc1a6d5a None admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:43.908870 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ba1924-80c9-45a7-8a78-0de4fc1a6d5a None admin] RBAC: Adding query filter params (name=service) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:27:43.909178 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ba1924-80c9-45a7-8a78-0de4fc1a6d5a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:27:43.910720 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ba1924-80c9-45a7-8a78-0de4fc1a6d5a None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:27:43.920276 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 307/632] 10.222.0.57 () {56 vars in 1103 bytes} [Fri Jul 1 16:27:43 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:27:43.921097 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ba1924-80c9-45a7-8a78-0de4fc1a6d5a None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:27:43.932909 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97da463-9d41-475b-b219-26cc3cdb2ee6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:27:43.933712 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97da463-9d41-475b-b219-26cc3cdb2ee6 None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:43.933986 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97da463-9d41-475b-b219-26cc3cdb2ee6 None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:43.934261 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97da463-9d41-475b-b219-26cc3cdb2ee6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:43.937662 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=100 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:27:43.942267 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d97da463-9d41-475b-b219-26cc3cdb2ee6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:27:43.943814 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d97da463-9d41-475b-b219-26cc3cdb2ee6 None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:27:43.958773 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application [None req-d97da463-9d41-475b-b219-26cc3cdb2ee6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 01 16:27:43.958773 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 16:27:43.958773 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 16:27:43.958773 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 16:27:43.958773 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 16:27:43.958773 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 16:27:43.958773 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 16:27:43.958773 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 16:27:43.958773 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 16:27:43.958773 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 16:27:43.958773 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 16:27:43.958773 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 16:27:43.958773 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 01 16:27:43.958773 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 01 16:27:43.958773 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 01 16:27:43.960042 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 01 16:27:43.960042 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 16:27:43.960042 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 16:27:43.960042 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 16:27:43.960042 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 16:27:43.960042 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 16:27:43.960042 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 16:27:43.960042 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 16:27:43.960042 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application with Lock( Jul 01 16:27:43.960042 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 16:27:43.960042 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self._enter() Jul 01 16:27:43.960042 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 16:27:43.960042 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 16:27:43.960042 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 16:27:43.960042 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.creator() Jul 01 16:27:43.960042 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 16:27:43.960042 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application created_value = creator( Jul 01 16:27:43.960968 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 01 16:27:43.960968 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 01 16:27:43.960968 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 01 16:27:43.960968 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 01 16:27:43.960968 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 01 16:27:43.960968 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 01 16:27:43.960968 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 01 16:27:43.960968 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application Jul 01 16:27:43.960968 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 326/633] 10.222.0.57 () {56 vars in 1111 bytes} [Fri Jul 1 16:27:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 16:27:43.962463 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c860cc-e742-490c-afc2-2f38c05738d0 None None] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:43.962463 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c860cc-e742-490c-afc2-2f38c05738d0 None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:43.962463 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c860cc-e742-490c-afc2-2f38c05738d0 None None] PATH_INFO: `/` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:43.966624 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 327/634] 10.222.0.57 () {54 vars in 926 bytes} [Fri Jul 1 16:27:43 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 16:27:43.967705 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c860cc-e742-490c-afc2-2f38c05738d0 None None] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:27:43.996873 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:27:44.035747 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ba1924-80c9-45a7-8a78-0de4fc1a6d5a None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:27:44.074164 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7dc4265-643c-4cad-9694-3b79630b3f70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:27:44.074542 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c860cc-e742-490c-afc2-2f38c05738d0 None None] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:27:44.074988 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7dc4265-643c-4cad-9694-3b79630b3f70 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:44.075270 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7dc4265-643c-4cad-9694-3b79630b3f70 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:44.075549 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7dc4265-643c-4cad-9694-3b79630b3f70 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:44.083013 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7dc4265-643c-4cad-9694-3b79630b3f70 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:27:44.084482 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7dc4265-643c-4cad-9694-3b79630b3f70 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:27:44.099536 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application [None req-d7dc4265-643c-4cad-9694-3b79630b3f70 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 01 16:27:44.099536 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 16:27:44.099536 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 16:27:44.099536 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 16:27:44.099536 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 16:27:44.099536 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 16:27:44.099536 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 16:27:44.099536 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 16:27:44.099536 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 16:27:44.099536 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 16:27:44.099536 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 16:27:44.099536 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 16:27:44.099536 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 01 16:27:44.099536 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 01 16:27:44.099536 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 01 16:27:44.101053 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 01 16:27:44.101053 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 16:27:44.101053 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 16:27:44.101053 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 16:27:44.101053 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 16:27:44.101053 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 16:27:44.101053 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 16:27:44.101053 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 16:27:44.101053 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application with Lock( Jul 01 16:27:44.101053 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 16:27:44.101053 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._enter() Jul 01 16:27:44.101053 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 16:27:44.101053 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 16:27:44.101053 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 16:27:44.101053 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.creator() Jul 01 16:27:44.101053 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 16:27:44.101053 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application created_value = creator( Jul 01 16:27:44.102141 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 01 16:27:44.102141 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 01 16:27:44.102141 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 01 16:27:44.102141 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 01 16:27:44.102141 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 01 16:27:44.102141 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 01 16:27:44.102141 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 01 16:27:44.102141 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Jul 01 16:27:44.102141 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 308/635] 10.222.0.57 () {56 vars in 1111 bytes} [Fri Jul 1 16:27:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 16:27:44.103101 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7dc4265-643c-4cad-9694-3b79630b3f70 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:27:44.117419 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e9352d-ece3-4f9d-b298-dadef779ee78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:27:44.118306 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e9352d-ece3-4f9d-b298-dadef779ee78 None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:44.118617 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e9352d-ece3-4f9d-b298-dadef779ee78 None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:44.118893 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e9352d-ece3-4f9d-b298-dadef779ee78 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:44.133258 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e9352d-ece3-4f9d-b298-dadef779ee78 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:27:44.134929 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e9352d-ece3-4f9d-b298-dadef779ee78 None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:27:44.153959 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application [None req-c5e9352d-ece3-4f9d-b298-dadef779ee78 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 16:27:44.153959 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 16:27:44.153959 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 16:27:44.153959 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 16:27:44.153959 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 16:27:44.153959 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 16:27:44.153959 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 16:27:44.153959 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 16:27:44.153959 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 16:27:44.153959 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 16:27:44.153959 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 16:27:44.153959 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 16:27:44.153959 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 01 16:27:44.153959 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 01 16:27:44.153959 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 01 16:27:44.155041 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 01 16:27:44.155041 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 16:27:44.155041 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 16:27:44.155041 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 16:27:44.155041 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 16:27:44.155041 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 16:27:44.155041 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 16:27:44.155041 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 16:27:44.155041 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application with Lock( Jul 01 16:27:44.155041 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 16:27:44.155041 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self._enter() Jul 01 16:27:44.155041 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 16:27:44.155041 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 16:27:44.155041 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 16:27:44.155041 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.creator() Jul 01 16:27:44.155041 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 16:27:44.155041 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application created_value = creator( Jul 01 16:27:44.155907 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 01 16:27:44.155907 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 01 16:27:44.155907 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 16:27:44.155907 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application Jul 01 16:27:44.155907 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 328/636] 10.222.0.57 () {56 vars in 1108 bytes} [Fri Jul 1 16:27:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 16:27:44.156760 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd8b58c-1bb9-45c6-ae9c-b6e6b918d339 None None] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:44.156760 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd8b58c-1bb9-45c6-ae9c-b6e6b918d339 None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:44.156917 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd8b58c-1bb9-45c6-ae9c-b6e6b918d339 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:44.175428 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.auth.core [None req-ffd8b58c-1bb9-45c6-ae9c-b6e6b918d339 None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80312) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:27:44.176596 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-ffd8b58c-1bb9-45c6-ae9c-b6e6b918d339 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:27:44.217760 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7dc4265-643c-4cad-9694-3b79630b3f70 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:27:44.223499 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=100 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:27:44.247954 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:27:44.253434 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 329/637] 10.222.0.57 () {58 vars in 1023 bytes} [Fri Jul 1 16:27:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 01 16:27:44.261728 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dee6f8c-168d-4884-953d-63bf723c2499 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:27:44.262477 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dee6f8c-168d-4884-953d-63bf723c2499 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:44.262770 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dee6f8c-168d-4884-953d-63bf723c2499 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:44.263108 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dee6f8c-168d-4884-953d-63bf723c2499 None admin] PATH_INFO: `/v3/projects` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:44.263881 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd8b58c-1bb9-45c6-ae9c-b6e6b918d339 None None] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:27:44.264008 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dee6f8c-168d-4884-953d-63bf723c2499 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:27:44.264215 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dee6f8c-168d-4884-953d-63bf723c2499 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:27:44.265770 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dee6f8c-168d-4884-953d-63bf723c2499 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:27:44.279348 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 309/638] 10.222.0.57 () {56 vars in 1148 bytes} [Fri Jul 1 16:27:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:27:44.280161 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dee6f8c-168d-4884-953d-63bf723c2499 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:27:44.399511 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd8b58c-1bb9-45c6-ae9c-b6e6b918d339 None None] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:27:44.399719 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dee6f8c-168d-4884-953d-63bf723c2499 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:27:44.436420 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc77e148-fefc-477f-8b46-e1b5e2482e70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:27:44.437087 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc77e148-fefc-477f-8b46-e1b5e2482e70 None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:44.437373 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc77e148-fefc-477f-8b46-e1b5e2482e70 None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:44.437649 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc77e148-fefc-477f-8b46-e1b5e2482e70 None admin] PATH_INFO: `/v3/projects` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:44.438475 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc77e148-fefc-477f-8b46-e1b5e2482e70 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80312) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:27:44.438788 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc77e148-fefc-477f-8b46-e1b5e2482e70 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:27:44.439795 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4de496-86e2-4f41-8716-e03b4f5c2a63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:27:44.440243 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc77e148-fefc-477f-8b46-e1b5e2482e70 None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:27:44.440459 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4de496-86e2-4f41-8716-e03b4f5c2a63 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:44.440746 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4de496-86e2-4f41-8716-e03b4f5c2a63 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:44.441026 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4de496-86e2-4f41-8716-e03b4f5c2a63 None admin] PATH_INFO: `/v3/domains` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:44.441753 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d4de496-86e2-4f41-8716-e03b4f5c2a63 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:27:44.442063 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d4de496-86e2-4f41-8716-e03b4f5c2a63 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:27:44.443449 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d4de496-86e2-4f41-8716-e03b4f5c2a63 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:27:44.452220 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 330/639] 10.222.0.57 () {56 vars in 1148 bytes} [Fri Jul 1 16:27:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:27:44.453019 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc77e148-fefc-477f-8b46-e1b5e2482e70 None admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:27:44.460092 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 310/640] 10.222.0.57 () {56 vars in 1109 bytes} [Fri Jul 1 16:27:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:27:44.460092 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4de496-86e2-4f41-8716-e03b4f5c2a63 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:27:44.494246 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=100 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:27:44.503997 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:27:44.554419 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc77e148-fefc-477f-8b46-e1b5e2482e70 None admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:27:44.560222 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4de496-86e2-4f41-8716-e03b4f5c2a63 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:27:44.587722 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405ef86a-db5c-4aa3-b880-7be34a8ff88b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:27:44.588510 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405ef86a-db5c-4aa3-b880-7be34a8ff88b None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:44.588787 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405ef86a-db5c-4aa3-b880-7be34a8ff88b None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:44.589060 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405ef86a-db5c-4aa3-b880-7be34a8ff88b None admin] PATH_INFO: `/v3/roles/service` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:44.593529 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e302d3-3fc8-4d94-9efd-c2285bb63c4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:27:44.594452 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e302d3-3fc8-4d94-9efd-c2285bb63c4f None admin] REQUEST_METHOD: `PUT` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:44.594736 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e302d3-3fc8-4d94-9efd-c2285bb63c4f None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:44.595017 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e302d3-3fc8-4d94-9efd-c2285bb63c4f None admin] PATH_INFO: `/v3/projects/e0788e6725a84e57a4174284c1845755/users/859fe410c2d2450fb803c6244839f27c/roles/e8809b67c9dc471daa1c2bd42c9a2baf` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:44.598964 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405ef86a-db5c-4aa3-b880-7be34a8ff88b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 01 16:27:44.607200 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application [None req-405ef86a-db5c-4aa3-b880-7be34a8ff88b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 01 16:27:44.607200 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 16:27:44.607200 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 16:27:44.607200 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 16:27:44.607200 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 16:27:44.607200 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 16:27:44.607200 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 16:27:44.607200 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 16:27:44.607200 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 16:27:44.607200 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 16:27:44.607200 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 16:27:44.607200 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 16:27:44.607200 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 01 16:27:44.607200 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 01 16:27:44.607200 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 01 16:27:44.607200 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 01 16:27:44.608199 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 01 16:27:44.608199 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 01 16:27:44.608199 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 01 16:27:44.608199 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 01 16:27:44.608199 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 16:27:44.608199 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 16:27:44.608199 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 16:27:44.608199 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 16:27:44.608199 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 16:27:44.608199 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 16:27:44.608199 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 16:27:44.608199 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application with Lock( Jul 01 16:27:44.608199 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 16:27:44.608199 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self._enter() Jul 01 16:27:44.608199 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 16:27:44.608199 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 16:27:44.609057 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 16:27:44.609057 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.creator() Jul 01 16:27:44.609057 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 16:27:44.609057 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application created_value = creator( Jul 01 16:27:44.609057 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 01 16:27:44.609057 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 01 16:27:44.609057 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 01 16:27:44.609057 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 01 16:27:44.609057 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 01 16:27:44.609057 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 01 16:27:44.609057 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 01 16:27:44.609057 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application Jul 01 16:27:44.609057 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 331/641] 10.222.0.57 () {56 vars in 1102 bytes} [Fri Jul 1 16:27:44 2022] GET /identity/v3/roles/service => generated 85 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 16:27:44.610370 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96beb690-d4d6-4c27-898e-7eb83e012594 None None] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:44.610370 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96beb690-d4d6-4c27-898e-7eb83e012594 None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:44.610628 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96beb690-d4d6-4c27-898e-7eb83e012594 None None] PATH_INFO: `/` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:44.611699 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e302d3-3fc8-4d94-9efd-c2285bb63c4f None admin] RBAC: Authorizing `identity:create_grant(project_id=e0788e6725a84e57a4174284c1845755, user_id=859fe410c2d2450fb803c6244839f27c, role_id=e8809b67c9dc471daa1c2bd42c9a2baf)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:27:44.612098 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 332/642] 10.222.0.57 () {54 vars in 926 bytes} [Fri Jul 1 16:27:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 16:27:44.613273 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e302d3-3fc8-4d94-9efd-c2285bb63c4f None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:27:44.614007 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96beb690-d4d6-4c27-898e-7eb83e012594 None None] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:27:44.629023 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 311/643] 10.222.0.57 () {58 vars in 1439 bytes} [Fri Jul 1 16:27:44 2022] PUT /identity/v3/projects/e0788e6725a84e57a4174284c1845755/users/859fe410c2d2450fb803c6244839f27c/roles/e8809b67c9dc471daa1c2bd42c9a2baf => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 16:27:44.629696 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e302d3-3fc8-4d94-9efd-c2285bb63c4f None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:27:44.727214 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=100 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:27:44.733464 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96beb690-d4d6-4c27-898e-7eb83e012594 None None] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:27:44.759918 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:27:44.767602 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e302d3-3fc8-4d94-9efd-c2285bb63c4f None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:27:44.777673 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0d7627-4ca9-4281-8f9d-9daf56c884ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:27:44.778368 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0d7627-4ca9-4281-8f9d-9daf56c884ec None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:44.778688 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0d7627-4ca9-4281-8f9d-9daf56c884ec None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:44.778969 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0d7627-4ca9-4281-8f9d-9daf56c884ec None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:44.779807 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a0d7627-4ca9-4281-8f9d-9daf56c884ec None admin] RBAC: Adding query filter params (role.id=e8809b67c9dc471daa1c2bd42c9a2baf, scope.project.id=e0788e6725a84e57a4174284c1845755, user.id=7a50f745b31941b7a7265c0fc852f48e) {{(pid=80312) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:27:44.780123 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a0d7627-4ca9-4281-8f9d-9daf56c884ec None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:27:44.782197 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a0d7627-4ca9-4281-8f9d-9daf56c884ec None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:27:44.792674 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 333/644] 10.222.0.57 () {56 vars in 1374 bytes} [Fri Jul 1 16:27:44 2022] GET /identity/v3/role_assignments?user.id=7a50f745b31941b7a7265c0fc852f48e&scope.project.id=e0788e6725a84e57a4174284c1845755&role.id=e8809b67c9dc471daa1c2bd42c9a2baf => generated 608 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:27:44.793516 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0d7627-4ca9-4281-8f9d-9daf56c884ec None admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:27:44.816617 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0880d28-39f6-4047-8147-aff0c9098fdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:27:44.817531 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0880d28-39f6-4047-8147-aff0c9098fdd None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:44.817820 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0880d28-39f6-4047-8147-aff0c9098fdd None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:44.818095 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0880d28-39f6-4047-8147-aff0c9098fdd None admin] PATH_INFO: `/v3/users/glance` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:44.828616 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0880d28-39f6-4047-8147-aff0c9098fdd None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:27:44.830442 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0880d28-39f6-4047-8147-aff0c9098fdd None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:27:44.848585 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application [None req-b0880d28-39f6-4047-8147-aff0c9098fdd None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 01 16:27:44.848585 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 16:27:44.848585 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 16:27:44.848585 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 16:27:44.848585 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 16:27:44.848585 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 16:27:44.848585 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 16:27:44.848585 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 16:27:44.848585 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 16:27:44.848585 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 16:27:44.848585 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 16:27:44.848585 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 16:27:44.848585 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 01 16:27:44.848585 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 01 16:27:44.848585 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 01 16:27:44.848585 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 01 16:27:44.849556 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 16:27:44.849556 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 16:27:44.849556 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 01 16:27:44.849556 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 01 16:27:44.849556 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 01 16:27:44.849556 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 01 16:27:44.849556 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 16:27:44.849556 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 16:27:44.849556 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 16:27:44.849556 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 16:27:44.849556 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 16:27:44.849556 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application with Lock( Jul 01 16:27:44.849556 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 16:27:44.849556 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._enter() Jul 01 16:27:44.849556 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 16:27:44.849556 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 16:27:44.849556 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 16:27:44.849556 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.creator() Jul 01 16:27:44.850726 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 16:27:44.850726 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application created_value = creator( Jul 01 16:27:44.850726 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 01 16:27:44.850726 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 01 16:27:44.850726 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 01 16:27:44.850726 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 01 16:27:44.850726 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 01 16:27:44.850726 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 01 16:27:44.850726 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 01 16:27:44.850726 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Jul 01 16:27:44.850726 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 312/645] 10.222.0.57 () {56 vars in 1099 bytes} [Fri Jul 1 16:27:44 2022] GET /identity/v3/users/glance => generated 84 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 16:27:44.855834 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70907098-8bdf-4059-826a-d1776c2dbb6b None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:44.856125 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70907098-8bdf-4059-826a-d1776c2dbb6b None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:44.856409 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70907098-8bdf-4059-826a-d1776c2dbb6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:44.874821 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-70907098-8bdf-4059-826a-d1776c2dbb6b None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:27:44.876042 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-70907098-8bdf-4059-826a-d1776c2dbb6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:27:44.945557 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0d7627-4ca9-4281-8f9d-9daf56c884ec None admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:27:44.955305 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 313/646] 10.222.0.57 () {58 vars in 1023 bytes} [Fri Jul 1 16:27:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 01 16:27:44.956166 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70907098-8bdf-4059-826a-d1776c2dbb6b None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:27:44.996275 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb509f6-0ed7-448d-92a9-0b8cf0bfc635 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:27:44.997257 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb509f6-0ed7-448d-92a9-0b8cf0bfc635 None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:44.997700 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb509f6-0ed7-448d-92a9-0b8cf0bfc635 None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:44.998398 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb509f6-0ed7-448d-92a9-0b8cf0bfc635 None admin] PATH_INFO: `/v3/roles` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:45.002822 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb509f6-0ed7-448d-92a9-0b8cf0bfc635 None admin] RBAC: Adding query filter params (name=service) {{(pid=80312) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:27:45.002822 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb509f6-0ed7-448d-92a9-0b8cf0bfc635 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:27:45.002822 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb509f6-0ed7-448d-92a9-0b8cf0bfc635 None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:27:45.014124 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 334/647] 10.222.0.57 () {56 vars in 1103 bytes} [Fri Jul 1 16:27:44 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:27:45.014939 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb509f6-0ed7-448d-92a9-0b8cf0bfc635 None admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:27:45.044716 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=100 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:27:45.086486 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:27:45.105014 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70907098-8bdf-4059-826a-d1776c2dbb6b None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:27:45.143561 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aac6c93-7184-4fd2-8edd-e0f359a98884 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:27:45.144444 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aac6c93-7184-4fd2-8edd-e0f359a98884 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:45.144889 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aac6c93-7184-4fd2-8edd-e0f359a98884 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:45.145175 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aac6c93-7184-4fd2-8edd-e0f359a98884 None admin] PATH_INFO: `/v3/users` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:45.146026 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aac6c93-7184-4fd2-8edd-e0f359a98884 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:27:45.146354 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aac6c93-7184-4fd2-8edd-e0f359a98884 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:27:45.148099 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aac6c93-7184-4fd2-8edd-e0f359a98884 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:27:45.164887 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb509f6-0ed7-448d-92a9-0b8cf0bfc635 None admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:27:45.173762 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 314/648] 10.222.0.57 () {56 vars in 1137 bytes} [Fri Jul 1 16:27:44 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:27:45.174653 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aac6c93-7184-4fd2-8edd-e0f359a98884 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:27:45.218698 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575a9080-8a27-401c-9208-e2e533e1fce1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:27:45.219512 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575a9080-8a27-401c-9208-e2e533e1fce1 None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:45.219798 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575a9080-8a27-401c-9208-e2e533e1fce1 None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:45.220087 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575a9080-8a27-401c-9208-e2e533e1fce1 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:45.232965 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-575a9080-8a27-401c-9208-e2e533e1fce1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 01 16:27:45.241912 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application [None req-575a9080-8a27-401c-9208-e2e533e1fce1 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 01 16:27:45.241912 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 16:27:45.241912 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 16:27:45.241912 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 16:27:45.241912 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 16:27:45.241912 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 16:27:45.241912 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 16:27:45.241912 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 16:27:45.241912 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 16:27:45.241912 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 16:27:45.241912 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 16:27:45.241912 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 16:27:45.241912 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 01 16:27:45.241912 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 01 16:27:45.241912 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 01 16:27:45.241912 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 01 16:27:45.243172 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 01 16:27:45.243172 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 01 16:27:45.243172 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 01 16:27:45.243172 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 01 16:27:45.243172 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 16:27:45.243172 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 16:27:45.243172 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 16:27:45.243172 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 16:27:45.243172 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 16:27:45.243172 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 16:27:45.243172 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 16:27:45.243172 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application with Lock( Jul 01 16:27:45.243172 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 16:27:45.243172 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self._enter() Jul 01 16:27:45.243172 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 16:27:45.243172 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 16:27:45.244143 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 16:27:45.244143 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.creator() Jul 01 16:27:45.244143 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 16:27:45.244143 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application created_value = creator( Jul 01 16:27:45.244143 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 01 16:27:45.244143 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 01 16:27:45.244143 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 01 16:27:45.244143 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 01 16:27:45.244143 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 01 16:27:45.244143 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 01 16:27:45.244143 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 01 16:27:45.244143 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application Jul 01 16:27:45.244143 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 335/649] 10.222.0.57 () {56 vars in 1102 bytes} [Fri Jul 1 16:27:45 2022] GET /identity/v3/roles/service => generated 85 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 16:27:45.244833 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575a9080-8a27-401c-9208-e2e533e1fce1 None admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:27:45.330541 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aac6c93-7184-4fd2-8edd-e0f359a98884 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:27:45.360111 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=100 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:27:45.371279 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34f8507-c433-4f23-9c12-7a23691c0811 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:27:45.372192 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34f8507-c433-4f23-9c12-7a23691c0811 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:45.372419 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34f8507-c433-4f23-9c12-7a23691c0811 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:45.372773 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34f8507-c433-4f23-9c12-7a23691c0811 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:45.385646 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f34f8507-c433-4f23-9c12-7a23691c0811 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:27:45.387405 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f34f8507-c433-4f23-9c12-7a23691c0811 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:27:45.390912 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:27:45.402106 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575a9080-8a27-401c-9208-e2e533e1fce1 None admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:27:45.413193 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application [None req-f34f8507-c433-4f23-9c12-7a23691c0811 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 16:27:45.413193 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 16:27:45.413193 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 16:27:45.413193 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 16:27:45.413193 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 16:27:45.413193 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 16:27:45.413193 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 16:27:45.413193 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 16:27:45.413193 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 16:27:45.413193 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 16:27:45.413193 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 16:27:45.413193 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 16:27:45.413193 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 01 16:27:45.413193 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 01 16:27:45.413193 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 01 16:27:45.414522 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 01 16:27:45.414522 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 16:27:45.414522 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 16:27:45.414522 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 16:27:45.414522 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 16:27:45.414522 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 16:27:45.414522 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 16:27:45.414522 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 16:27:45.414522 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application with Lock( Jul 01 16:27:45.414522 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 16:27:45.414522 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._enter() Jul 01 16:27:45.414522 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 16:27:45.414522 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 16:27:45.414522 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 16:27:45.414522 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.creator() Jul 01 16:27:45.414522 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 16:27:45.414522 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application created_value = creator( Jul 01 16:27:45.415441 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 01 16:27:45.415441 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 01 16:27:45.415441 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 16:27:45.415441 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Jul 01 16:27:45.415441 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 315/650] 10.222.0.57 () {56 vars in 1108 bytes} [Fri Jul 1 16:27:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 16:27:45.416172 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34f8507-c433-4f23-9c12-7a23691c0811 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:27:45.449399 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ad0921-65be-4066-a9b2-e0d969bdf3f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:27:45.450940 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ad0921-65be-4066-a9b2-e0d969bdf3f4 None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:45.451511 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ad0921-65be-4066-a9b2-e0d969bdf3f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:45.451999 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ad0921-65be-4066-a9b2-e0d969bdf3f4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:45.547300 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34f8507-c433-4f23-9c12-7a23691c0811 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:27:45.551603 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ad0921-65be-4066-a9b2-e0d969bdf3f4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:27:45.553295 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ad0921-65be-4066-a9b2-e0d969bdf3f4 None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:27:45.590111 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ccdfc8-511d-4835-85cc-45e386fb0417 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:27:45.590931 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ccdfc8-511d-4835-85cc-45e386fb0417 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:45.591321 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ccdfc8-511d-4835-85cc-45e386fb0417 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:45.591756 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ccdfc8-511d-4835-85cc-45e386fb0417 None admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:45.592924 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ccdfc8-511d-4835-85cc-45e386fb0417 None admin] RBAC: Adding query filter params (name=service) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:27:45.593211 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ccdfc8-511d-4835-85cc-45e386fb0417 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:27:45.594875 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ccdfc8-511d-4835-85cc-45e386fb0417 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:27:45.605676 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 316/651] 10.222.0.57 () {56 vars in 1103 bytes} [Fri Jul 1 16:27:45 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:27:45.606854 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ccdfc8-511d-4835-85cc-45e386fb0417 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:27:45.660753 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=100 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:27:45.683676 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 336/652] 10.222.0.57 () {58 vars in 1282 bytes} [Fri Jul 1 16:27:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 439 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 01 16:27:45.685474 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ad0921-65be-4066-a9b2-e0d969bdf3f4 None admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:27:45.689907 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:27:45.747785 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ccdfc8-511d-4835-85cc-45e386fb0417 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:27:45.791587 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e492cd16-f5e7-47de-adf6-85515ce2ed4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:27:45.793181 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e492cd16-f5e7-47de-adf6-85515ce2ed4e None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:45.793743 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e492cd16-f5e7-47de-adf6-85515ce2ed4e None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:45.794230 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e492cd16-f5e7-47de-adf6-85515ce2ed4e None admin] PATH_INFO: `/v3/domains` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:45.795311 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e492cd16-f5e7-47de-adf6-85515ce2ed4e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:27:45.796102 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e492cd16-f5e7-47de-adf6-85515ce2ed4e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:27:45.798048 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e492cd16-f5e7-47de-adf6-85515ce2ed4e None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:27:45.801960 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ad0921-65be-4066-a9b2-e0d969bdf3f4 None admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:27:45.818466 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 317/653] 10.222.0.57 () {56 vars in 1109 bytes} [Fri Jul 1 16:27:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:27:45.818466 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e492cd16-f5e7-47de-adf6-85515ce2ed4e None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:27:45.860317 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29abf54-e032-407a-bbf1-72a90df95b06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:27:45.861406 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29abf54-e032-407a-bbf1-72a90df95b06 None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:45.861852 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29abf54-e032-407a-bbf1-72a90df95b06 None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:45.862262 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29abf54-e032-407a-bbf1-72a90df95b06 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:45.878954 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b29abf54-e032-407a-bbf1-72a90df95b06 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:27:45.880617 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b29abf54-e032-407a-bbf1-72a90df95b06 None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:27:45.900123 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application [None req-b29abf54-e032-407a-bbf1-72a90df95b06 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 16:27:45.900123 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 16:27:45.900123 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 16:27:45.900123 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 16:27:45.900123 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 16:27:45.900123 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 16:27:45.900123 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 16:27:45.900123 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 16:27:45.900123 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 16:27:45.900123 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 16:27:45.900123 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 16:27:45.900123 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 16:27:45.900123 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 01 16:27:45.900123 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 01 16:27:45.900123 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 01 16:27:45.901328 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 01 16:27:45.901328 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 16:27:45.901328 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 16:27:45.901328 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 16:27:45.901328 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 16:27:45.901328 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 16:27:45.901328 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 16:27:45.901328 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 16:27:45.901328 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application with Lock( Jul 01 16:27:45.901328 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 16:27:45.901328 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self._enter() Jul 01 16:27:45.901328 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 16:27:45.901328 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 16:27:45.901328 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 16:27:45.901328 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.creator() Jul 01 16:27:45.901328 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 16:27:45.901328 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application created_value = creator( Jul 01 16:27:45.902406 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 01 16:27:45.902406 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 01 16:27:45.902406 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 16:27:45.902406 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application Jul 01 16:27:45.904255 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 337/654] 10.222.0.57 () {56 vars in 1108 bytes} [Fri Jul 1 16:27:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 16:27:45.905184 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29abf54-e032-407a-bbf1-72a90df95b06 None admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:27:45.945795 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=100 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:27:45.961315 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e492cd16-f5e7-47de-adf6-85515ce2ed4e None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:27:45.982180 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:27:46.002777 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6fb89cd-2231-4bb7-93d3-463dcd61cdaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:27:46.003795 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fb89cd-2231-4bb7-93d3-463dcd61cdaf None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:46.004245 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fb89cd-2231-4bb7-93d3-463dcd61cdaf None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:46.004709 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fb89cd-2231-4bb7-93d3-463dcd61cdaf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:46.018624 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6fb89cd-2231-4bb7-93d3-463dcd61cdaf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:27:46.020387 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6fb89cd-2231-4bb7-93d3-463dcd61cdaf None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:27:46.034003 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29abf54-e032-407a-bbf1-72a90df95b06 None admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:27:46.051614 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application [None req-c6fb89cd-2231-4bb7-93d3-463dcd61cdaf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 16:27:46.051614 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 16:27:46.051614 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 16:27:46.051614 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 16:27:46.051614 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 16:27:46.051614 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 16:27:46.051614 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 16:27:46.051614 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 16:27:46.051614 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 16:27:46.051614 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 16:27:46.051614 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 16:27:46.051614 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 16:27:46.051614 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 01 16:27:46.051614 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 01 16:27:46.051614 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 01 16:27:46.052577 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 01 16:27:46.052577 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 16:27:46.052577 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 16:27:46.052577 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 16:27:46.052577 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 16:27:46.052577 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 16:27:46.052577 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 16:27:46.052577 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 16:27:46.052577 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application with Lock( Jul 01 16:27:46.052577 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 16:27:46.052577 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._enter() Jul 01 16:27:46.052577 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 16:27:46.052577 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 16:27:46.052577 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 16:27:46.052577 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.creator() Jul 01 16:27:46.052577 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 16:27:46.052577 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application created_value = creator( Jul 01 16:27:46.053669 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 01 16:27:46.053669 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 01 16:27:46.053669 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 16:27:46.053669 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Jul 01 16:27:46.053669 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 318/655] 10.222.0.57 () {56 vars in 1108 bytes} [Fri Jul 1 16:27:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 16:27:46.054075 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6fb89cd-2231-4bb7-93d3-463dcd61cdaf None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:27:46.079389 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34c5c48-5217-4935-a037-d8734e0bd40d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:27:46.080801 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34c5c48-5217-4935-a037-d8734e0bd40d None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:46.081273 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34c5c48-5217-4935-a037-d8734e0bd40d None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:46.081706 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34c5c48-5217-4935-a037-d8734e0bd40d None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:46.093336 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a34c5c48-5217-4935-a037-d8734e0bd40d None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:27:46.095679 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a34c5c48-5217-4935-a037-d8734e0bd40d None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:27:46.115218 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application [None req-a34c5c48-5217-4935-a037-d8734e0bd40d None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 01 16:27:46.115218 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 16:27:46.115218 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 16:27:46.115218 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 16:27:46.115218 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 16:27:46.115218 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 16:27:46.115218 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 16:27:46.115218 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 16:27:46.115218 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 16:27:46.115218 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 16:27:46.115218 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 16:27:46.115218 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 16:27:46.115218 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 01 16:27:46.115218 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 01 16:27:46.115218 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 01 16:27:46.115218 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 01 16:27:46.116366 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 16:27:46.116366 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 16:27:46.116366 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 01 16:27:46.116366 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 01 16:27:46.116366 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 01 16:27:46.116366 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 01 16:27:46.116366 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 16:27:46.116366 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 16:27:46.116366 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 16:27:46.116366 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 16:27:46.116366 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 16:27:46.116366 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application with Lock( Jul 01 16:27:46.116366 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 16:27:46.116366 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self._enter() Jul 01 16:27:46.116366 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 16:27:46.116366 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 16:27:46.116366 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 16:27:46.116366 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.creator() Jul 01 16:27:46.117311 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 16:27:46.117311 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application created_value = creator( Jul 01 16:27:46.117311 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 01 16:27:46.117311 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 01 16:27:46.117311 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 01 16:27:46.117311 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 01 16:27:46.117311 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 01 16:27:46.117311 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 01 16:27:46.117311 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 01 16:27:46.117311 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application Jul 01 16:27:46.119726 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 338/656] 10.222.0.57 () {56 vars in 1099 bytes} [Fri Jul 1 16:27:45 2022] GET /identity/v3/users/cinder => generated 84 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 16:27:46.120653 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34c5c48-5217-4935-a037-d8734e0bd40d None admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:27:46.183752 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6fb89cd-2231-4bb7-93d3-463dcd61cdaf None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:27:46.220353 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31a1eea-dcb1-4728-a817-26e6904dabd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:27:46.221176 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31a1eea-dcb1-4728-a817-26e6904dabd3 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:46.221565 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31a1eea-dcb1-4728-a817-26e6904dabd3 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:46.221969 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31a1eea-dcb1-4728-a817-26e6904dabd3 None admin] PATH_INFO: `/v3/domains` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:46.222841 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b31a1eea-dcb1-4728-a817-26e6904dabd3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:27:46.223255 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b31a1eea-dcb1-4728-a817-26e6904dabd3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:27:46.224886 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b31a1eea-dcb1-4728-a817-26e6904dabd3 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:27:46.232847 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34c5c48-5217-4935-a037-d8734e0bd40d None admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:27:46.238675 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=100 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:27:46.239726 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 319/657] 10.222.0.57 () {56 vars in 1109 bytes} [Fri Jul 1 16:27:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:27:46.240673 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31a1eea-dcb1-4728-a817-26e6904dabd3 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:27:46.273630 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:27:46.281351 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec1f0d1-b045-4c2d-8aec-345cac4dbc2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:27:46.282266 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec1f0d1-b045-4c2d-8aec-345cac4dbc2f None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:46.282722 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec1f0d1-b045-4c2d-8aec-345cac4dbc2f None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:46.283149 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec1f0d1-b045-4c2d-8aec-345cac4dbc2f None admin] PATH_INFO: `/v3/domains` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:46.284021 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec1f0d1-b045-4c2d-8aec-345cac4dbc2f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80312) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:27:46.284497 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec1f0d1-b045-4c2d-8aec-345cac4dbc2f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:27:46.286004 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec1f0d1-b045-4c2d-8aec-345cac4dbc2f None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:27:46.303129 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 339/658] 10.222.0.57 () {56 vars in 1109 bytes} [Fri Jul 1 16:27:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:27:46.304897 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec1f0d1-b045-4c2d-8aec-345cac4dbc2f None admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:27:46.355036 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31a1eea-dcb1-4728-a817-26e6904dabd3 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:27:46.391470 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f936d54c-e782-4da1-bb8a-07d6b6bba050 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:27:46.392164 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f936d54c-e782-4da1-bb8a-07d6b6bba050 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:46.392524 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f936d54c-e782-4da1-bb8a-07d6b6bba050 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:46.392906 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f936d54c-e782-4da1-bb8a-07d6b6bba050 None admin] PATH_INFO: `/v3/users` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:46.393807 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f936d54c-e782-4da1-bb8a-07d6b6bba050 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:27:46.394406 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f936d54c-e782-4da1-bb8a-07d6b6bba050 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:27:46.395834 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f936d54c-e782-4da1-bb8a-07d6b6bba050 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:27:46.413968 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec1f0d1-b045-4c2d-8aec-345cac4dbc2f None admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:27:46.419817 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 320/659] 10.222.0.57 () {56 vars in 1137 bytes} [Fri Jul 1 16:27:46 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:27:46.420861 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f936d54c-e782-4da1-bb8a-07d6b6bba050 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:27:46.452887 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589f3495-c8ba-4667-8a02-e3ead8a29d6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:27:46.454254 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589f3495-c8ba-4667-8a02-e3ead8a29d6f None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:46.454888 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589f3495-c8ba-4667-8a02-e3ead8a29d6f None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:46.455345 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589f3495-c8ba-4667-8a02-e3ead8a29d6f None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:46.464449 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-589f3495-c8ba-4667-8a02-e3ead8a29d6f None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:27:46.466179 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-589f3495-c8ba-4667-8a02-e3ead8a29d6f None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:27:46.482100 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application [None req-589f3495-c8ba-4667-8a02-e3ead8a29d6f None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 01 16:27:46.482100 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 16:27:46.482100 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 16:27:46.482100 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 16:27:46.482100 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 16:27:46.482100 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 16:27:46.482100 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 16:27:46.482100 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 16:27:46.482100 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 16:27:46.482100 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 16:27:46.482100 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 16:27:46.482100 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 16:27:46.482100 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 01 16:27:46.482100 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 01 16:27:46.482100 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 01 16:27:46.482100 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 01 16:27:46.483109 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 16:27:46.483109 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 16:27:46.483109 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 01 16:27:46.483109 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 01 16:27:46.483109 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 01 16:27:46.483109 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 01 16:27:46.483109 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 16:27:46.483109 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 16:27:46.483109 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 16:27:46.483109 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 16:27:46.483109 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 16:27:46.483109 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application with Lock( Jul 01 16:27:46.483109 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 16:27:46.483109 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self._enter() Jul 01 16:27:46.483109 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 16:27:46.483109 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 16:27:46.483109 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 16:27:46.483109 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.creator() Jul 01 16:27:46.484054 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 16:27:46.484054 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application created_value = creator( Jul 01 16:27:46.484054 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 01 16:27:46.484054 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 01 16:27:46.484054 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 01 16:27:46.484054 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 01 16:27:46.484054 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 01 16:27:46.484054 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 01 16:27:46.484054 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 01 16:27:46.484054 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application Jul 01 16:27:46.486929 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 340/660] 10.222.0.57 () {56 vars in 1102 bytes} [Fri Jul 1 16:27:46 2022] GET /identity/v3/users/neutron => generated 85 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 16:27:46.487971 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589f3495-c8ba-4667-8a02-e3ead8a29d6f None admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:27:46.508503 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=100 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:27:46.523979 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:27:46.538468 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f936d54c-e782-4da1-bb8a-07d6b6bba050 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:27:46.585387 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c8b26c-085f-49b9-934c-283d4945ce34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:27:46.587766 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c8b26c-085f-49b9-934c-283d4945ce34 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:46.588147 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c8b26c-085f-49b9-934c-283d4945ce34 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:46.588506 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c8b26c-085f-49b9-934c-283d4945ce34 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:46.601896 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c8b26c-085f-49b9-934c-283d4945ce34 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:27:46.603537 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c8b26c-085f-49b9-934c-283d4945ce34 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:27:46.605489 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589f3495-c8ba-4667-8a02-e3ead8a29d6f None admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:27:46.624755 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application [None req-64c8b26c-085f-49b9-934c-283d4945ce34 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 01 16:27:46.624755 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 16:27:46.624755 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 16:27:46.624755 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 16:27:46.624755 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 16:27:46.624755 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 16:27:46.624755 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 16:27:46.624755 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 16:27:46.624755 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 16:27:46.624755 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 16:27:46.624755 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 16:27:46.624755 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 16:27:46.624755 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 01 16:27:46.624755 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 01 16:27:46.624755 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 01 16:27:46.625733 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 01 16:27:46.625733 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 16:27:46.625733 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 16:27:46.625733 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 16:27:46.625733 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 16:27:46.625733 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 16:27:46.625733 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 16:27:46.625733 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 16:27:46.625733 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application with Lock( Jul 01 16:27:46.625733 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 16:27:46.625733 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._enter() Jul 01 16:27:46.625733 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 16:27:46.625733 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 16:27:46.625733 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 16:27:46.625733 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.creator() Jul 01 16:27:46.625733 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 16:27:46.625733 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application created_value = creator( Jul 01 16:27:46.626738 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 01 16:27:46.626738 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 01 16:27:46.626738 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 01 16:27:46.626738 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 01 16:27:46.626738 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 01 16:27:46.626738 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 01 16:27:46.626738 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 01 16:27:46.626738 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Jul 01 16:27:46.631060 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 321/661] 10.222.0.57 () {56 vars in 1111 bytes} [Fri Jul 1 16:27:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 16:27:46.632279 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c8b26c-085f-49b9-934c-283d4945ce34 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:27:46.647522 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4393e9f-8085-46fd-957d-4cbd3745703c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:27:46.648644 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4393e9f-8085-46fd-957d-4cbd3745703c None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:46.649224 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4393e9f-8085-46fd-957d-4cbd3745703c None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:46.649680 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4393e9f-8085-46fd-957d-4cbd3745703c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:46.734062 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4393e9f-8085-46fd-957d-4cbd3745703c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:27:46.735988 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4393e9f-8085-46fd-957d-4cbd3745703c None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:27:46.746430 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c8b26c-085f-49b9-934c-283d4945ce34 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:27:46.756497 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=100 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:27:46.782477 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd215ec-a60e-40d5-9985-632d4f1a00c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:27:46.783575 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd215ec-a60e-40d5-9985-632d4f1a00c1 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:46.783575 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd215ec-a60e-40d5-9985-632d4f1a00c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:46.783793 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd215ec-a60e-40d5-9985-632d4f1a00c1 None admin] PATH_INFO: `/v3/users` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:46.784744 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd215ec-a60e-40d5-9985-632d4f1a00c1 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:27:46.785126 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd215ec-a60e-40d5-9985-632d4f1a00c1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:27:46.786678 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd215ec-a60e-40d5-9985-632d4f1a00c1 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:27:46.806377 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 322/662] 10.222.0.57 () {56 vars in 1139 bytes} [Fri Jul 1 16:27:46 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:27:46.807496 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd215ec-a60e-40d5-9985-632d4f1a00c1 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:27:46.816379 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:27:46.864525 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 341/663] 10.222.0.57 () {58 vars in 1282 bytes} [Fri Jul 1 16:27:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 01 16:27:46.865656 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4393e9f-8085-46fd-957d-4cbd3745703c None admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:27:46.920196 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd215ec-a60e-40d5-9985-632d4f1a00c1 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:27:46.963083 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800e9a72-3498-4e70-9759-3fde3f52dc43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:27:46.964262 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800e9a72-3498-4e70-9759-3fde3f52dc43 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:46.964557 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800e9a72-3498-4e70-9759-3fde3f52dc43 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:46.964843 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800e9a72-3498-4e70-9759-3fde3f52dc43 None admin] PATH_INFO: `/v3/projects` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:46.965651 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-800e9a72-3498-4e70-9759-3fde3f52dc43 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:27:46.965986 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-800e9a72-3498-4e70-9759-3fde3f52dc43 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:27:46.967439 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-800e9a72-3498-4e70-9759-3fde3f52dc43 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:27:46.980220 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4393e9f-8085-46fd-957d-4cbd3745703c None admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:27:46.981477 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 323/664] 10.222.0.57 () {56 vars in 1148 bytes} [Fri Jul 1 16:27:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:27:46.982819 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553372d2-17c9-4374-9400-2cda281734ee None None] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:46.983108 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553372d2-17c9-4374-9400-2cda281734ee None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:46.983387 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553372d2-17c9-4374-9400-2cda281734ee None None] PATH_INFO: `/` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:46.984785 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 324/665] 10.222.0.57 () {54 vars in 926 bytes} [Fri Jul 1 16:27:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 16:27:46.989318 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553372d2-17c9-4374-9400-2cda281734ee None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:27:47.006100 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=100 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:27:47.029434 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd42b9a7-71a4-4387-abe2-2fb7c73eae14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:27:47.030703 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd42b9a7-71a4-4387-abe2-2fb7c73eae14 None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:47.031988 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd42b9a7-71a4-4387-abe2-2fb7c73eae14 None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:47.032465 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd42b9a7-71a4-4387-abe2-2fb7c73eae14 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:47.064074 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:27:47.101735 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553372d2-17c9-4374-9400-2cda281734ee None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:27:47.110738 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd42b9a7-71a4-4387-abe2-2fb7c73eae14 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:27:47.112204 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd42b9a7-71a4-4387-abe2-2fb7c73eae14 None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:27:47.135269 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1eab549-575b-4dc1-81f2-06659b341177 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:27:47.136053 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1eab549-575b-4dc1-81f2-06659b341177 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:47.136337 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1eab549-575b-4dc1-81f2-06659b341177 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:47.136617 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1eab549-575b-4dc1-81f2-06659b341177 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:47.146467 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1eab549-575b-4dc1-81f2-06659b341177 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:27:47.148487 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1eab549-575b-4dc1-81f2-06659b341177 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:27:47.166561 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application [None req-a1eab549-575b-4dc1-81f2-06659b341177 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 16:27:47.166561 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 16:27:47.166561 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 16:27:47.166561 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 16:27:47.166561 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 16:27:47.166561 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 16:27:47.166561 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 16:27:47.166561 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 16:27:47.166561 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 16:27:47.166561 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 16:27:47.166561 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 16:27:47.166561 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 16:27:47.166561 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 01 16:27:47.166561 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 01 16:27:47.166561 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 01 16:27:47.167494 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 01 16:27:47.167494 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 16:27:47.167494 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 16:27:47.167494 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 16:27:47.167494 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 16:27:47.167494 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 16:27:47.167494 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 16:27:47.167494 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 16:27:47.167494 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application with Lock( Jul 01 16:27:47.167494 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 16:27:47.167494 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._enter() Jul 01 16:27:47.167494 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 16:27:47.167494 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 16:27:47.167494 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 16:27:47.167494 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.creator() Jul 01 16:27:47.167494 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 16:27:47.167494 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application created_value = creator( Jul 01 16:27:47.168380 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 01 16:27:47.168380 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 01 16:27:47.168380 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 16:27:47.168380 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Jul 01 16:27:47.168380 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 325/666] 10.222.0.57 () {56 vars in 1108 bytes} [Fri Jul 1 16:27:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 16:27:47.168665 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1eab549-575b-4dc1-81f2-06659b341177 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:27:47.216909 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 342/667] 10.222.0.57 () {58 vars in 1282 bytes} [Fri Jul 1 16:27:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 01 16:27:47.218117 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86150126-614a-49cb-b58f-eed71b20a3a7 None None] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:47.218499 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86150126-614a-49cb-b58f-eed71b20a3a7 None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:47.218781 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86150126-614a-49cb-b58f-eed71b20a3a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:47.235299 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.auth.core [None req-86150126-614a-49cb-b58f-eed71b20a3a7 None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80312) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:27:47.236599 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-86150126-614a-49cb-b58f-eed71b20a3a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:27:47.248807 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=100 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:27:47.272900 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1eab549-575b-4dc1-81f2-06659b341177 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:27:47.289540 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:27:47.305099 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 343/668] 10.222.0.57 () {58 vars in 1023 bytes} [Fri Jul 1 16:27:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 01 16:27:47.306029 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a34e210-ecd6-4da5-8a8d-c5369519bc77 None None] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:47.306360 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a34e210-ecd6-4da5-8a8d-c5369519bc77 None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:47.306620 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a34e210-ecd6-4da5-8a8d-c5369519bc77 None None] PATH_INFO: `/` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:47.310485 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4deac165-9a43-411e-9a2e-1be972ce9c09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:27:47.310631 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 344/669] 10.222.0.57 () {54 vars in 926 bytes} [Fri Jul 1 16:27:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 16:27:47.310710 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4deac165-9a43-411e-9a2e-1be972ce9c09 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:47.310782 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a34e210-ecd6-4da5-8a8d-c5369519bc77 None None] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:27:47.310852 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4deac165-9a43-411e-9a2e-1be972ce9c09 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:47.310852 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4deac165-9a43-411e-9a2e-1be972ce9c09 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:47.310852 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4deac165-9a43-411e-9a2e-1be972ce9c09 None admin] RBAC: Adding query filter params (role.id=e8809b67c9dc471daa1c2bd42c9a2baf, scope.project.id=e0788e6725a84e57a4174284c1845755, user.id=fd028715bfea43099e7c6cde429ecf92) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:27:47.310852 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4deac165-9a43-411e-9a2e-1be972ce9c09 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:27:47.311762 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4deac165-9a43-411e-9a2e-1be972ce9c09 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:27:47.318983 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 326/670] 10.222.0.57 () {56 vars in 1374 bytes} [Fri Jul 1 16:27:47 2022] GET /identity/v3/role_assignments?user.id=fd028715bfea43099e7c6cde429ecf92&scope.project.id=e0788e6725a84e57a4174284c1845755&role.id=e8809b67c9dc471daa1c2bd42c9a2baf => generated 608 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:27:47.320280 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4deac165-9a43-411e-9a2e-1be972ce9c09 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:27:47.443002 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a34e210-ecd6-4da5-8a8d-c5369519bc77 None None] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:27:47.446750 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4deac165-9a43-411e-9a2e-1be972ce9c09 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:27:47.484680 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3b3340-13bd-47e4-9b6f-813234b2dbb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:27:47.485268 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3b3340-13bd-47e4-9b6f-813234b2dbb2 None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:47.485555 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3b3340-13bd-47e4-9b6f-813234b2dbb2 None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:47.485654 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dae750b-bd61-44d2-88fa-58b01b044709 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:27:47.485864 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3b3340-13bd-47e4-9b6f-813234b2dbb2 None admin] PATH_INFO: `/v3/domains` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:47.486418 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dae750b-bd61-44d2-88fa-58b01b044709 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:47.486597 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e3b3340-13bd-47e4-9b6f-813234b2dbb2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80312) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:27:47.486691 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dae750b-bd61-44d2-88fa-58b01b044709 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:47.486911 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e3b3340-13bd-47e4-9b6f-813234b2dbb2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:27:47.487006 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dae750b-bd61-44d2-88fa-58b01b044709 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:47.488308 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e3b3340-13bd-47e4-9b6f-813234b2dbb2 None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:27:47.500186 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dae750b-bd61-44d2-88fa-58b01b044709 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:27:47.501669 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dae750b-bd61-44d2-88fa-58b01b044709 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:27:47.506337 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 345/671] 10.222.0.57 () {56 vars in 1109 bytes} [Fri Jul 1 16:27:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:27:47.508345 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c48cfa-13de-440d-b4c4-dac79cf4c91a None None] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:47.508635 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c48cfa-13de-440d-b4c4-dac79cf4c91a None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:47.508916 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c48cfa-13de-440d-b4c4-dac79cf4c91a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:47.522235 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application [None req-4dae750b-bd61-44d2-88fa-58b01b044709 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 16:27:47.522235 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 16:27:47.522235 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 16:27:47.522235 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 16:27:47.522235 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 16:27:47.522235 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 16:27:47.522235 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 16:27:47.522235 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 16:27:47.522235 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 16:27:47.522235 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 16:27:47.522235 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 16:27:47.522235 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 16:27:47.522235 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 01 16:27:47.522235 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 01 16:27:47.522235 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 01 16:27:47.523314 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 01 16:27:47.523314 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 16:27:47.523314 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 16:27:47.523314 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 16:27:47.523314 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 16:27:47.523314 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 16:27:47.523314 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 16:27:47.523314 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 16:27:47.523314 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application with Lock( Jul 01 16:27:47.523314 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 16:27:47.523314 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._enter() Jul 01 16:27:47.523314 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 16:27:47.523314 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 16:27:47.523314 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 16:27:47.523314 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.creator() Jul 01 16:27:47.523314 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 16:27:47.523314 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application created_value = creator( Jul 01 16:27:47.524286 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 01 16:27:47.524286 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 01 16:27:47.524286 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 16:27:47.524286 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Jul 01 16:27:47.524286 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 327/672] 10.222.0.57 () {56 vars in 1108 bytes} [Fri Jul 1 16:27:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 16:27:47.524714 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dae750b-bd61-44d2-88fa-58b01b044709 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:27:47.525240 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=100 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:27:47.534804 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.auth.core [None req-31c48cfa-13de-440d-b4c4-dac79cf4c91a None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80312) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:27:47.536472 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-31c48cfa-13de-440d-b4c4-dac79cf4c91a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:27:47.563367 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:27:47.612362 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 346/673] 10.222.0.57 () {58 vars in 1023 bytes} [Fri Jul 1 16:27:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 01 16:27:47.613556 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c48cfa-13de-440d-b4c4-dac79cf4c91a None None] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:27:47.649861 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dae750b-bd61-44d2-88fa-58b01b044709 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:27:47.687440 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bdc54cd-a5db-427e-9770-8c5c8964fbe0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:27:47.688477 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bdc54cd-a5db-427e-9770-8c5c8964fbe0 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:47.688905 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bdc54cd-a5db-427e-9770-8c5c8964fbe0 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:47.689180 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bdc54cd-a5db-427e-9770-8c5c8964fbe0 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:47.700648 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bdc54cd-a5db-427e-9770-8c5c8964fbe0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 01 16:27:47.708542 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application [None req-8bdc54cd-a5db-427e-9770-8c5c8964fbe0 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 01 16:27:47.708542 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 16:27:47.708542 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 16:27:47.708542 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 16:27:47.708542 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 16:27:47.708542 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 16:27:47.708542 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 16:27:47.708542 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 16:27:47.708542 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 16:27:47.708542 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 16:27:47.708542 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 16:27:47.708542 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 16:27:47.708542 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 01 16:27:47.708542 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 01 16:27:47.708542 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 01 16:27:47.708542 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 01 16:27:47.709711 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 01 16:27:47.709711 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 01 16:27:47.709711 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 01 16:27:47.709711 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 01 16:27:47.709711 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 16:27:47.709711 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 16:27:47.709711 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 16:27:47.709711 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 16:27:47.709711 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 16:27:47.709711 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 16:27:47.709711 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 16:27:47.709711 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application with Lock( Jul 01 16:27:47.709711 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 16:27:47.709711 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._enter() Jul 01 16:27:47.709711 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 16:27:47.709711 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 16:27:47.712225 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 16:27:47.712225 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.creator() Jul 01 16:27:47.712225 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 16:27:47.712225 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application created_value = creator( Jul 01 16:27:47.712225 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 01 16:27:47.712225 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 01 16:27:47.712225 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 01 16:27:47.712225 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 01 16:27:47.712225 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 01 16:27:47.712225 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 01 16:27:47.712225 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 01 16:27:47.712225 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Jul 01 16:27:47.712225 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 328/674] 10.222.0.57 () {56 vars in 1102 bytes} [Fri Jul 1 16:27:47 2022] GET /identity/v3/roles/service => generated 85 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 16:27:47.712225 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bdc54cd-a5db-427e-9770-8c5c8964fbe0 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:27:47.730827 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c48cfa-13de-440d-b4c4-dac79cf4c91a None None] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:27:47.769535 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c484af-23df-48e6-a0b4-d7ac35f4d17b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:27:47.770251 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c484af-23df-48e6-a0b4-d7ac35f4d17b None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:47.770567 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c484af-23df-48e6-a0b4-d7ac35f4d17b None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:47.770841 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c484af-23df-48e6-a0b4-d7ac35f4d17b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:47.774325 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=100 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:27:47.781265 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c484af-23df-48e6-a0b4-d7ac35f4d17b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:27:47.782738 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c484af-23df-48e6-a0b4-d7ac35f4d17b None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:27:47.796839 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application [None req-b0c484af-23df-48e6-a0b4-d7ac35f4d17b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 01 16:27:47.796839 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 16:27:47.796839 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 16:27:47.796839 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 16:27:47.796839 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 16:27:47.796839 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 16:27:47.796839 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 16:27:47.796839 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 16:27:47.796839 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 16:27:47.796839 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 16:27:47.796839 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 16:27:47.796839 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 16:27:47.796839 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 01 16:27:47.796839 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 01 16:27:47.796839 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 01 16:27:47.797803 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 01 16:27:47.797803 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 16:27:47.797803 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 16:27:47.797803 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 16:27:47.797803 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 16:27:47.797803 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 16:27:47.797803 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 16:27:47.797803 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 16:27:47.797803 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application with Lock( Jul 01 16:27:47.797803 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 16:27:47.797803 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self._enter() Jul 01 16:27:47.797803 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 16:27:47.797803 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 16:27:47.797803 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 16:27:47.797803 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.creator() Jul 01 16:27:47.797803 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 16:27:47.797803 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application created_value = creator( Jul 01 16:27:47.803385 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 01 16:27:47.803385 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 01 16:27:47.803385 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 01 16:27:47.803385 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 01 16:27:47.803385 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 01 16:27:47.803385 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 01 16:27:47.803385 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 01 16:27:47.803385 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application Jul 01 16:27:47.803385 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 347/675] 10.222.0.57 () {56 vars in 1111 bytes} [Fri Jul 1 16:27:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 16:27:47.803385 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c484af-23df-48e6-a0b4-d7ac35f4d17b None admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:27:47.828930 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:27:47.842223 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bdc54cd-a5db-427e-9770-8c5c8964fbe0 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:27:47.882395 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aeaa1fe-a65e-4930-b912-5087843ce288 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:27:47.883346 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aeaa1fe-a65e-4930-b912-5087843ce288 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:47.883633 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aeaa1fe-a65e-4930-b912-5087843ce288 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:47.883913 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aeaa1fe-a65e-4930-b912-5087843ce288 None admin] PATH_INFO: `/v3/domains` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:47.884659 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aeaa1fe-a65e-4930-b912-5087843ce288 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:27:47.884969 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aeaa1fe-a65e-4930-b912-5087843ce288 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:27:47.886357 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aeaa1fe-a65e-4930-b912-5087843ce288 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:27:47.899292 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 329/676] 10.222.0.57 () {56 vars in 1109 bytes} [Fri Jul 1 16:27:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:27:47.900131 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aeaa1fe-a65e-4930-b912-5087843ce288 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:27:47.912501 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c484af-23df-48e6-a0b4-d7ac35f4d17b None admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:27:47.952833 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68aa11c-622d-4817-a923-9046057caf7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:27:47.953631 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68aa11c-622d-4817-a923-9046057caf7d None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:47.954170 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68aa11c-622d-4817-a923-9046057caf7d None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:47.954480 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68aa11c-622d-4817-a923-9046057caf7d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:47.965546 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c68aa11c-622d-4817-a923-9046057caf7d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 01 16:27:47.973346 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application [None req-c68aa11c-622d-4817-a923-9046057caf7d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 01 16:27:47.973346 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 16:27:47.973346 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 16:27:47.973346 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 16:27:47.973346 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 16:27:47.973346 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 16:27:47.973346 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 16:27:47.973346 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 16:27:47.973346 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 16:27:47.973346 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 16:27:47.973346 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 16:27:47.973346 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 16:27:47.973346 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 01 16:27:47.973346 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 01 16:27:47.973346 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 01 16:27:47.973346 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 01 16:27:47.974387 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 01 16:27:47.974387 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 01 16:27:47.974387 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 01 16:27:47.974387 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 01 16:27:47.974387 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 16:27:47.974387 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 16:27:47.974387 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 16:27:47.974387 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 16:27:47.974387 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 16:27:47.974387 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 16:27:47.974387 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 16:27:47.974387 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application with Lock( Jul 01 16:27:47.974387 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 16:27:47.974387 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self._enter() Jul 01 16:27:47.974387 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 16:27:47.974387 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 16:27:47.975820 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 16:27:47.975820 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.creator() Jul 01 16:27:47.975820 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 16:27:47.975820 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application created_value = creator( Jul 01 16:27:47.975820 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 01 16:27:47.975820 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 01 16:27:47.975820 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 01 16:27:47.975820 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 01 16:27:47.975820 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 01 16:27:47.975820 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 01 16:27:47.975820 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 01 16:27:47.975820 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application Jul 01 16:27:47.975820 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 348/677] 10.222.0.57 () {56 vars in 1096 bytes} [Fri Jul 1 16:27:47 2022] GET /identity/v3/roles/admin => generated 83 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 16:27:47.977399 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68aa11c-622d-4817-a923-9046057caf7d None admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:27:48.021166 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aeaa1fe-a65e-4930-b912-5087843ce288 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:27:48.040060 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=100 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:27:48.057983 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7217565e-3113-405a-bb4a-12d22090ec81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:27:48.058679 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7217565e-3113-405a-bb4a-12d22090ec81 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:48.058961 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7217565e-3113-405a-bb4a-12d22090ec81 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:48.059236 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7217565e-3113-405a-bb4a-12d22090ec81 None admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:48.059986 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7217565e-3113-405a-bb4a-12d22090ec81 None admin] RBAC: Adding query filter params (name=service) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:27:48.060293 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7217565e-3113-405a-bb4a-12d22090ec81 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:27:48.061652 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7217565e-3113-405a-bb4a-12d22090ec81 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:27:48.069968 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 330/678] 10.222.0.57 () {56 vars in 1103 bytes} [Fri Jul 1 16:27:47 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:27:48.070983 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7217565e-3113-405a-bb4a-12d22090ec81 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:27:48.091454 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:27:48.101089 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68aa11c-622d-4817-a923-9046057caf7d None admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:27:48.137052 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7833b8c-f4c9-41dc-a51a-cd5235b5f0c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:27:48.137720 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7833b8c-f4c9-41dc-a51a-cd5235b5f0c4 None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:48.138005 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7833b8c-f4c9-41dc-a51a-cd5235b5f0c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:48.138278 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7833b8c-f4c9-41dc-a51a-cd5235b5f0c4 None admin] PATH_INFO: `/v3/projects` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:48.139104 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7833b8c-f4c9-41dc-a51a-cd5235b5f0c4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80312) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:27:48.139412 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7833b8c-f4c9-41dc-a51a-cd5235b5f0c4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:27:48.140849 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7833b8c-f4c9-41dc-a51a-cd5235b5f0c4 None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:27:48.158376 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 349/679] 10.222.0.57 () {56 vars in 1148 bytes} [Fri Jul 1 16:27:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:27:48.158376 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7833b8c-f4c9-41dc-a51a-cd5235b5f0c4 None admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:27:48.187434 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7217565e-3113-405a-bb4a-12d22090ec81 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:27:48.224118 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb279755-6cd5-4601-9d07-a2319408a700 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:27:48.224910 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb279755-6cd5-4601-9d07-a2319408a700 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:48.225984 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb279755-6cd5-4601-9d07-a2319408a700 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:48.226303 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb279755-6cd5-4601-9d07-a2319408a700 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:48.233549 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb279755-6cd5-4601-9d07-a2319408a700 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:27:48.235055 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb279755-6cd5-4601-9d07-a2319408a700 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:27:48.250791 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application [None req-fb279755-6cd5-4601-9d07-a2319408a700 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 01 16:27:48.250791 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 16:27:48.250791 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 16:27:48.250791 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 16:27:48.250791 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 16:27:48.250791 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 16:27:48.250791 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 16:27:48.250791 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 16:27:48.250791 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 16:27:48.250791 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 16:27:48.250791 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 16:27:48.250791 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 16:27:48.250791 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 01 16:27:48.250791 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 01 16:27:48.250791 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 01 16:27:48.251723 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 01 16:27:48.251723 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 16:27:48.251723 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 16:27:48.251723 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 16:27:48.251723 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 16:27:48.251723 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 16:27:48.251723 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 16:27:48.251723 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 16:27:48.251723 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application with Lock( Jul 01 16:27:48.251723 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 16:27:48.251723 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._enter() Jul 01 16:27:48.251723 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 16:27:48.251723 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 16:27:48.251723 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 16:27:48.251723 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.creator() Jul 01 16:27:48.251723 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 16:27:48.251723 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application created_value = creator( Jul 01 16:27:48.252649 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 01 16:27:48.252649 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 01 16:27:48.252649 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 01 16:27:48.252649 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 01 16:27:48.252649 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 01 16:27:48.252649 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 01 16:27:48.252649 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 01 16:27:48.252649 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Jul 01 16:27:48.252649 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 331/680] 10.222.0.57 () {56 vars in 1111 bytes} [Fri Jul 1 16:27:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 16:27:48.253148 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb279755-6cd5-4601-9d07-a2319408a700 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:27:48.268712 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7833b8c-f4c9-41dc-a51a-cd5235b5f0c4 None admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:27:48.288896 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=100 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:27:48.308261 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884aec7e-8ca2-4ab9-9a0d-fba9f4061698 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:27:48.308930 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884aec7e-8ca2-4ab9-9a0d-fba9f4061698 None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:48.309216 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884aec7e-8ca2-4ab9-9a0d-fba9f4061698 None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:48.309494 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884aec7e-8ca2-4ab9-9a0d-fba9f4061698 None admin] PATH_INFO: `/v3/roles` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:48.310237 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-884aec7e-8ca2-4ab9-9a0d-fba9f4061698 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80312) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:27:48.310578 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-884aec7e-8ca2-4ab9-9a0d-fba9f4061698 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:27:48.311942 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-884aec7e-8ca2-4ab9-9a0d-fba9f4061698 None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:27:48.322324 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 350/681] 10.222.0.57 () {56 vars in 1099 bytes} [Fri Jul 1 16:27:48 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:27:48.323240 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884aec7e-8ca2-4ab9-9a0d-fba9f4061698 None admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:27:48.342965 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:27:48.367080 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb279755-6cd5-4601-9d07-a2319408a700 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:27:48.401343 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b683ba7-edeb-4fb4-a800-62948e7c03fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:27:48.402285 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b683ba7-edeb-4fb4-a800-62948e7c03fe None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:48.402595 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b683ba7-edeb-4fb4-a800-62948e7c03fe None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:48.402877 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b683ba7-edeb-4fb4-a800-62948e7c03fe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:48.416701 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b683ba7-edeb-4fb4-a800-62948e7c03fe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:27:48.418600 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b683ba7-edeb-4fb4-a800-62948e7c03fe None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:27:48.435524 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884aec7e-8ca2-4ab9-9a0d-fba9f4061698 None admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:27:48.436947 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application [None req-9b683ba7-edeb-4fb4-a800-62948e7c03fe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 16:27:48.436947 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 16:27:48.436947 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 16:27:48.436947 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 16:27:48.436947 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 16:27:48.436947 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 16:27:48.436947 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 16:27:48.436947 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 16:27:48.436947 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 16:27:48.436947 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 16:27:48.436947 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 16:27:48.436947 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 16:27:48.436947 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 01 16:27:48.436947 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 01 16:27:48.436947 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 01 16:27:48.437984 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 01 16:27:48.437984 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 16:27:48.437984 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 16:27:48.437984 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 16:27:48.437984 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 16:27:48.437984 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 16:27:48.437984 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 16:27:48.437984 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 16:27:48.437984 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application with Lock( Jul 01 16:27:48.437984 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 16:27:48.437984 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._enter() Jul 01 16:27:48.437984 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 16:27:48.437984 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 16:27:48.437984 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 16:27:48.437984 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.creator() Jul 01 16:27:48.437984 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 16:27:48.437984 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application created_value = creator( Jul 01 16:27:48.443798 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 01 16:27:48.443798 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 01 16:27:48.443798 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 16:27:48.443798 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Jul 01 16:27:48.443798 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 332/682] 10.222.0.57 () {56 vars in 1108 bytes} [Fri Jul 1 16:27:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 16:27:48.443798 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b683ba7-edeb-4fb4-a800-62948e7c03fe None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:27:48.482840 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca47b99-1e7b-4aa1-9271-f3cd89a5972e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:27:48.483502 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca47b99-1e7b-4aa1-9271-f3cd89a5972e None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:48.483786 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca47b99-1e7b-4aa1-9271-f3cd89a5972e None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:48.484154 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca47b99-1e7b-4aa1-9271-f3cd89a5972e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:48.484909 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca47b99-1e7b-4aa1-9271-f3cd89a5972e None admin] RBAC: Adding query filter params (role.id=e8809b67c9dc471daa1c2bd42c9a2baf, scope.project.id=e0788e6725a84e57a4174284c1845755, user.id=dbddf971a2354bbdbb9da1d63f3abf88) {{(pid=80312) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:27:48.485219 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca47b99-1e7b-4aa1-9271-f3cd89a5972e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:27:48.486705 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca47b99-1e7b-4aa1-9271-f3cd89a5972e None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:27:48.493796 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 351/683] 10.222.0.57 () {56 vars in 1374 bytes} [Fri Jul 1 16:27:48 2022] GET /identity/v3/role_assignments?user.id=dbddf971a2354bbdbb9da1d63f3abf88&scope.project.id=e0788e6725a84e57a4174284c1845755&role.id=e8809b67c9dc471daa1c2bd42c9a2baf => generated 608 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:27:48.494610 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca47b99-1e7b-4aa1-9271-f3cd89a5972e None admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:27:48.556717 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=100 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:27:48.585673 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b683ba7-edeb-4fb4-a800-62948e7c03fe None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:27:48.617838 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:27:48.628234 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e107c31d-0748-48ad-9fae-b8982b431737 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:27:48.629200 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e107c31d-0748-48ad-9fae-b8982b431737 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:48.629652 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e107c31d-0748-48ad-9fae-b8982b431737 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:48.630083 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e107c31d-0748-48ad-9fae-b8982b431737 None admin] PATH_INFO: `/v3/projects` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:48.631136 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e107c31d-0748-48ad-9fae-b8982b431737 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:27:48.632299 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e107c31d-0748-48ad-9fae-b8982b431737 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:27:48.634297 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e107c31d-0748-48ad-9fae-b8982b431737 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:27:48.641647 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca47b99-1e7b-4aa1-9271-f3cd89a5972e None admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:27:48.654663 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 333/684] 10.222.0.57 () {56 vars in 1148 bytes} [Fri Jul 1 16:27:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:27:48.655949 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e107c31d-0748-48ad-9fae-b8982b431737 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:27:48.689724 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0fd6c04-2543-4dd6-ab79-ad0067a927b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:27:48.690814 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fd6c04-2543-4dd6-ab79-ad0067a927b6 None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:48.691511 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fd6c04-2543-4dd6-ab79-ad0067a927b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:48.691797 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fd6c04-2543-4dd6-ab79-ad0067a927b6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:48.706703 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0fd6c04-2543-4dd6-ab79-ad0067a927b6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:27:48.708232 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0fd6c04-2543-4dd6-ab79-ad0067a927b6 None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:27:48.729698 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application [None req-e0fd6c04-2543-4dd6-ab79-ad0067a927b6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 16:27:48.729698 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 16:27:48.729698 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 16:27:48.729698 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 16:27:48.729698 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 16:27:48.729698 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 16:27:48.729698 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 16:27:48.729698 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 16:27:48.729698 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 16:27:48.729698 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 16:27:48.729698 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 16:27:48.729698 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 16:27:48.729698 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 01 16:27:48.729698 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 01 16:27:48.729698 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 01 16:27:48.730717 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 01 16:27:48.730717 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 16:27:48.730717 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 16:27:48.730717 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 16:27:48.730717 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 16:27:48.730717 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 16:27:48.730717 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 16:27:48.730717 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 16:27:48.730717 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application with Lock( Jul 01 16:27:48.730717 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 16:27:48.730717 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self._enter() Jul 01 16:27:48.730717 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 16:27:48.730717 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 16:27:48.730717 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 16:27:48.730717 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.creator() Jul 01 16:27:48.730717 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 16:27:48.730717 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application created_value = creator( Jul 01 16:27:48.732243 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 01 16:27:48.732243 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 01 16:27:48.732243 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 16:27:48.732243 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application Jul 01 16:27:48.732243 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 352/685] 10.222.0.57 () {56 vars in 1108 bytes} [Fri Jul 1 16:27:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 16:27:48.732243 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0fd6c04-2543-4dd6-ab79-ad0067a927b6 None admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:27:48.782097 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e107c31d-0748-48ad-9fae-b8982b431737 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:27:48.825413 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66751c82-ec38-407c-994a-c29e553b9d80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:27:48.827576 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66751c82-ec38-407c-994a-c29e553b9d80 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:48.828073 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66751c82-ec38-407c-994a-c29e553b9d80 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:48.828576 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66751c82-ec38-407c-994a-c29e553b9d80 None admin] PATH_INFO: `/v3/domains` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:48.829783 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66751c82-ec38-407c-994a-c29e553b9d80 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:27:48.830305 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66751c82-ec38-407c-994a-c29e553b9d80 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:27:48.831912 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66751c82-ec38-407c-994a-c29e553b9d80 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:27:48.856096 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 334/686] 10.222.0.57 () {56 vars in 1109 bytes} [Fri Jul 1 16:27:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:27:48.856096 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66751c82-ec38-407c-994a-c29e553b9d80 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:27:48.872072 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=100 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:27:48.880953 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0fd6c04-2543-4dd6-ab79-ad0067a927b6 None admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:27:48.918733 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:27:48.929128 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5fd231c-2fd7-4080-bc19-d5c8c4c49b85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:27:48.930660 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fd231c-2fd7-4080-bc19-d5c8c4c49b85 None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:48.931111 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fd231c-2fd7-4080-bc19-d5c8c4c49b85 None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:48.931536 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fd231c-2fd7-4080-bc19-d5c8c4c49b85 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:48.932987 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5fd231c-2fd7-4080-bc19-d5c8c4c49b85 None admin] RBAC: Adding query filter params (role.id=e8809b67c9dc471daa1c2bd42c9a2baf, scope.project.id=e0788e6725a84e57a4174284c1845755, user.id=4ed200ce29684c5f80354fa9a8638fe2) {{(pid=80312) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:27:48.933462 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5fd231c-2fd7-4080-bc19-d5c8c4c49b85 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:27:48.935543 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5fd231c-2fd7-4080-bc19-d5c8c4c49b85 None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:27:48.943960 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 353/687] 10.222.0.57 () {56 vars in 1374 bytes} [Fri Jul 1 16:27:48 2022] GET /identity/v3/role_assignments?user.id=4ed200ce29684c5f80354fa9a8638fe2&scope.project.id=e0788e6725a84e57a4174284c1845755&role.id=e8809b67c9dc471daa1c2bd42c9a2baf => generated 608 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:27:48.945037 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5fd231c-2fd7-4080-bc19-d5c8c4c49b85 None admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:27:49.015107 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66751c82-ec38-407c-994a-c29e553b9d80 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:27:49.069214 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5fd231c-2fd7-4080-bc19-d5c8c4c49b85 None admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:27:49.111065 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883921d0-5bad-44d6-b911-21c04010e9f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:27:49.112212 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883921d0-5bad-44d6-b911-21c04010e9f9 None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:49.112642 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883921d0-5bad-44d6-b911-21c04010e9f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:49.113076 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883921d0-5bad-44d6-b911-21c04010e9f9 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:49.115056 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02a49ef-3927-43a7-83cc-7d2ec7c1e515 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:27:49.115915 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02a49ef-3927-43a7-83cc-7d2ec7c1e515 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:49.116206 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02a49ef-3927-43a7-83cc-7d2ec7c1e515 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:49.116486 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02a49ef-3927-43a7-83cc-7d2ec7c1e515 None admin] PATH_INFO: `/v3/domains` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:49.117513 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02a49ef-3927-43a7-83cc-7d2ec7c1e515 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:27:49.117657 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02a49ef-3927-43a7-83cc-7d2ec7c1e515 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:27:49.119072 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02a49ef-3927-43a7-83cc-7d2ec7c1e515 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:27:49.123349 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-883921d0-5bad-44d6-b911-21c04010e9f9 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:27:49.125029 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-883921d0-5bad-44d6-b911-21c04010e9f9 None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:27:49.142806 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application [None req-883921d0-5bad-44d6-b911-21c04010e9f9 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 01 16:27:49.142806 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 16:27:49.142806 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 16:27:49.142806 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 16:27:49.142806 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 16:27:49.142806 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 16:27:49.142806 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 16:27:49.142806 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 16:27:49.142806 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 16:27:49.142806 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 16:27:49.142806 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 16:27:49.142806 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 16:27:49.142806 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 01 16:27:49.142806 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 01 16:27:49.142806 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 01 16:27:49.142806 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 01 16:27:49.143908 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 16:27:49.143908 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 16:27:49.143908 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 01 16:27:49.143908 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 01 16:27:49.143908 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 01 16:27:49.143908 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 01 16:27:49.143908 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 16:27:49.143908 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 16:27:49.143908 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 16:27:49.143908 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 16:27:49.143908 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 16:27:49.143908 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application with Lock( Jul 01 16:27:49.143908 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 16:27:49.143908 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self._enter() Jul 01 16:27:49.143908 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 16:27:49.143908 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 16:27:49.143908 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 16:27:49.143908 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.creator() Jul 01 16:27:49.144949 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 16:27:49.144949 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application created_value = creator( Jul 01 16:27:49.144949 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 01 16:27:49.144949 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 01 16:27:49.144949 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 01 16:27:49.144949 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 01 16:27:49.144949 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 01 16:27:49.144949 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 01 16:27:49.144949 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 01 16:27:49.144949 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application Jul 01 16:27:49.148853 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 354/688] 10.222.0.57 () {56 vars in 1096 bytes} [Fri Jul 1 16:27:48 2022] GET /identity/v3/users/swift => generated 83 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 16:27:49.152452 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 335/689] 10.222.0.57 () {56 vars in 1109 bytes} [Fri Jul 1 16:27:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:27:49.158250 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883921d0-5bad-44d6-b911-21c04010e9f9 None admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:27:49.163927 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02a49ef-3927-43a7-83cc-7d2ec7c1e515 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:27:49.182202 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=100 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:27:49.292008 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883921d0-5bad-44d6-b911-21c04010e9f9 None admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:27:49.329540 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02a49ef-3927-43a7-83cc-7d2ec7c1e515 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:27:49.343513 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023c7026-5a69-4465-a188-cf6fe3413716 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:27:49.344193 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023c7026-5a69-4465-a188-cf6fe3413716 None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:49.344483 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023c7026-5a69-4465-a188-cf6fe3413716 None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:49.344762 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023c7026-5a69-4465-a188-cf6fe3413716 None admin] PATH_INFO: `/v3/users` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:49.345335 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:27:49.345612 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-023c7026-5a69-4465-a188-cf6fe3413716 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80312) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:27:49.345925 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-023c7026-5a69-4465-a188-cf6fe3413716 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:27:49.347578 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-023c7026-5a69-4465-a188-cf6fe3413716 None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:27:49.374464 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 355/690] 10.222.0.57 () {56 vars in 1135 bytes} [Fri Jul 1 16:27:49 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:27:49.378085 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023c7026-5a69-4465-a188-cf6fe3413716 None admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:27:49.395885 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c30dd77-18d4-4250-a55d-3cab71d05a4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:27:49.398848 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c30dd77-18d4-4250-a55d-3cab71d05a4a None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:49.399580 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c30dd77-18d4-4250-a55d-3cab71d05a4a None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:49.400085 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c30dd77-18d4-4250-a55d-3cab71d05a4a None admin] PATH_INFO: `/v3/users/nova` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:49.412531 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c30dd77-18d4-4250-a55d-3cab71d05a4a None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:27:49.416087 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c30dd77-18d4-4250-a55d-3cab71d05a4a None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:27:49.435573 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application [None req-9c30dd77-18d4-4250-a55d-3cab71d05a4a None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 01 16:27:49.435573 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 16:27:49.435573 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 16:27:49.435573 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 16:27:49.435573 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 16:27:49.435573 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 16:27:49.435573 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 16:27:49.435573 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 16:27:49.435573 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 16:27:49.435573 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 16:27:49.435573 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 16:27:49.435573 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 16:27:49.435573 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 01 16:27:49.435573 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 01 16:27:49.435573 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 01 16:27:49.435573 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 01 16:27:49.436590 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 16:27:49.436590 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 16:27:49.436590 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 01 16:27:49.436590 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 01 16:27:49.436590 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 01 16:27:49.436590 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 01 16:27:49.436590 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 16:27:49.436590 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 16:27:49.436590 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 16:27:49.436590 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 16:27:49.436590 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 16:27:49.436590 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application with Lock( Jul 01 16:27:49.436590 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 16:27:49.436590 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._enter() Jul 01 16:27:49.436590 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 16:27:49.436590 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 16:27:49.436590 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 16:27:49.436590 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.creator() Jul 01 16:27:49.438213 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 16:27:49.438213 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application created_value = creator( Jul 01 16:27:49.438213 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 01 16:27:49.438213 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 01 16:27:49.438213 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 01 16:27:49.438213 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 01 16:27:49.438213 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 01 16:27:49.438213 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 01 16:27:49.438213 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 01 16:27:49.438213 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Jul 01 16:27:49.438213 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 336/691] 10.222.0.57 () {56 vars in 1093 bytes} [Fri Jul 1 16:27:49 2022] GET /identity/v3/users/nova => generated 82 bytes in 274 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 01 16:27:49.442492 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c30dd77-18d4-4250-a55d-3cab71d05a4a None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:27:49.483940 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=100 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:27:49.552537 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023c7026-5a69-4465-a188-cf6fe3413716 None admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:27:49.575307 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c30dd77-18d4-4250-a55d-3cab71d05a4a None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:27:49.614929 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7636d057-e76f-4050-b111-c616fa2513b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:27:49.616369 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7636d057-e76f-4050-b111-c616fa2513b8 None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:49.616655 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7636d057-e76f-4050-b111-c616fa2513b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:49.616934 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7636d057-e76f-4050-b111-c616fa2513b8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:49.620975 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552ba030-9773-4d71-955a-3b77d7270678 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:27:49.622223 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552ba030-9773-4d71-955a-3b77d7270678 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:49.622706 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552ba030-9773-4d71-955a-3b77d7270678 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:49.623155 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552ba030-9773-4d71-955a-3b77d7270678 None admin] PATH_INFO: `/v3/users` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:49.624170 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-552ba030-9773-4d71-955a-3b77d7270678 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:27:49.624730 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-552ba030-9773-4d71-955a-3b77d7270678 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:27:49.626480 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-552ba030-9773-4d71-955a-3b77d7270678 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:27:49.651274 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 337/692] 10.222.0.57 () {56 vars in 1133 bytes} [Fri Jul 1 16:27:49 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 01 16:27:49.653370 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f041881-db95-4bc7-9246-0a735ac9bb1c None None] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:49.653838 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f041881-db95-4bc7-9246-0a735ac9bb1c None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:49.654291 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f041881-db95-4bc7-9246-0a735ac9bb1c None None] PATH_INFO: `/` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:49.656041 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 338/693] 10.222.0.57 () {54 vars in 926 bytes} [Fri Jul 1 16:27:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 16:27:49.656930 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f041881-db95-4bc7-9246-0a735ac9bb1c None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:27:49.697894 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:27:49.708366 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7636d057-e76f-4050-b111-c616fa2513b8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:27:49.710635 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7636d057-e76f-4050-b111-c616fa2513b8 None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:27:49.815069 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f041881-db95-4bc7-9246-0a735ac9bb1c None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:27:49.824832 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=100 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:27:49.868753 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 356/694] 10.222.0.57 () {58 vars in 1282 bytes} [Fri Jul 1 16:27:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 487 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 01 16:27:49.871104 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb0fa1f-5922-4f71-84e7-72e0bfa54ba1 None None] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:49.871408 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb0fa1f-5922-4f71-84e7-72e0bfa54ba1 None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:49.871701 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb0fa1f-5922-4f71-84e7-72e0bfa54ba1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:49.873877 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee02f6d7-4874-4593-be1f-a680e2ff0056 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:27:49.874861 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee02f6d7-4874-4593-be1f-a680e2ff0056 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:49.875352 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee02f6d7-4874-4593-be1f-a680e2ff0056 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:49.875830 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee02f6d7-4874-4593-be1f-a680e2ff0056 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:49.889820 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.auth.core [None req-7eb0fa1f-5922-4f71-84e7-72e0bfa54ba1 None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80312) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:27:49.891280 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-7eb0fa1f-5922-4f71-84e7-72e0bfa54ba1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:27:49.965655 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee02f6d7-4874-4593-be1f-a680e2ff0056 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:27:49.967245 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee02f6d7-4874-4593-be1f-a680e2ff0056 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:27:49.991296 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 357/695] 10.222.0.57 () {58 vars in 1023 bytes} [Fri Jul 1 16:27:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 01 16:27:49.992865 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb0fa1f-5922-4f71-84e7-72e0bfa54ba1 None None] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:27:49.997270 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:27:50.098677 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 339/696] 10.222.0.57 () {58 vars in 1282 bytes} [Fri Jul 1 16:27:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 442 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 01 16:27:50.099697 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee02f6d7-4874-4593-be1f-a680e2ff0056 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:27:50.120725 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb0fa1f-5922-4f71-84e7-72e0bfa54ba1 None None] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:27:50.128976 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=100 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:27:50.158178 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3e3896-72df-4b4f-9339-4b4fb330bf9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:27:50.159406 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3e3896-72df-4b4f-9339-4b4fb330bf9f None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:50.159916 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3e3896-72df-4b4f-9339-4b4fb330bf9f None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:50.160400 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3e3896-72df-4b4f-9339-4b4fb330bf9f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:50.172302 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d3e3896-72df-4b4f-9339-4b4fb330bf9f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:27:50.174076 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d3e3896-72df-4b4f-9339-4b4fb330bf9f None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:27:50.192761 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application [None req-0d3e3896-72df-4b4f-9339-4b4fb330bf9f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 16:27:50.192761 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 16:27:50.192761 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 16:27:50.192761 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 16:27:50.192761 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 16:27:50.192761 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 16:27:50.192761 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 16:27:50.192761 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 16:27:50.192761 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 16:27:50.192761 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 16:27:50.192761 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 16:27:50.192761 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 16:27:50.192761 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 01 16:27:50.192761 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 01 16:27:50.192761 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 01 16:27:50.193815 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 01 16:27:50.193815 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 16:27:50.193815 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 16:27:50.193815 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 16:27:50.193815 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 16:27:50.193815 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 16:27:50.193815 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 16:27:50.193815 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 16:27:50.193815 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application with Lock( Jul 01 16:27:50.193815 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 16:27:50.193815 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self._enter() Jul 01 16:27:50.193815 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 16:27:50.193815 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 16:27:50.193815 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 16:27:50.193815 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.creator() Jul 01 16:27:50.193815 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 16:27:50.193815 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application created_value = creator( Jul 01 16:27:50.194989 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 01 16:27:50.194989 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 01 16:27:50.194989 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 16:27:50.194989 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application Jul 01 16:27:50.197273 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 358/697] 10.222.0.57 () {56 vars in 1108 bytes} [Fri Jul 1 16:27:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 16:27:50.198285 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3e3896-72df-4b4f-9339-4b4fb330bf9f None admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:27:50.221226 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee02f6d7-4874-4593-be1f-a680e2ff0056 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:27:50.251646 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:27:50.263579 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17f02a3-6084-46ed-8373-ca534ad6de28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:27:50.264328 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17f02a3-6084-46ed-8373-ca534ad6de28 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:50.264611 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17f02a3-6084-46ed-8373-ca534ad6de28 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:50.264891 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17f02a3-6084-46ed-8373-ca534ad6de28 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:50.276324 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c17f02a3-6084-46ed-8373-ca534ad6de28 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:27:50.277779 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c17f02a3-6084-46ed-8373-ca534ad6de28 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:27:50.297120 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application [None req-c17f02a3-6084-46ed-8373-ca534ad6de28 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 16:27:50.297120 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 16:27:50.297120 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 16:27:50.297120 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 16:27:50.297120 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 16:27:50.297120 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 16:27:50.297120 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 16:27:50.297120 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 16:27:50.297120 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 16:27:50.297120 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 16:27:50.297120 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 16:27:50.297120 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 16:27:50.297120 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 01 16:27:50.297120 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 01 16:27:50.297120 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 01 16:27:50.298977 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 01 16:27:50.298977 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 16:27:50.298977 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 16:27:50.298977 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 16:27:50.298977 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 16:27:50.298977 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 16:27:50.298977 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 16:27:50.298977 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 16:27:50.298977 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application with Lock( Jul 01 16:27:50.298977 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 16:27:50.298977 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._enter() Jul 01 16:27:50.298977 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 16:27:50.298977 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 16:27:50.298977 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 16:27:50.298977 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.creator() Jul 01 16:27:50.298977 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 16:27:50.298977 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application created_value = creator( Jul 01 16:27:50.300815 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 01 16:27:50.300815 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 01 16:27:50.300815 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 16:27:50.300815 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Jul 01 16:27:50.300815 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 340/698] 10.222.0.57 () {56 vars in 1108 bytes} [Fri Jul 1 16:27:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 16:27:50.300815 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17f02a3-6084-46ed-8373-ca534ad6de28 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:27:50.337965 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3e3896-72df-4b4f-9339-4b4fb330bf9f None admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:27:50.387773 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13bfa48-36ab-4bdc-bd5a-388c7deed4cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:27:50.389087 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13bfa48-36ab-4bdc-bd5a-388c7deed4cc None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:50.389768 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13bfa48-36ab-4bdc-bd5a-388c7deed4cc None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:50.390372 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13bfa48-36ab-4bdc-bd5a-388c7deed4cc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:50.402338 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b13bfa48-36ab-4bdc-bd5a-388c7deed4cc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:27:50.404156 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b13bfa48-36ab-4bdc-bd5a-388c7deed4cc None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:27:50.414649 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17f02a3-6084-46ed-8373-ca534ad6de28 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:27:50.424148 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application [None req-b13bfa48-36ab-4bdc-bd5a-388c7deed4cc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 16:27:50.424148 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 16:27:50.424148 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 16:27:50.424148 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 16:27:50.424148 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 16:27:50.424148 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 16:27:50.424148 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 16:27:50.424148 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 16:27:50.424148 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 16:27:50.424148 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 16:27:50.424148 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 16:27:50.424148 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 16:27:50.424148 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 01 16:27:50.424148 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 01 16:27:50.424148 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 01 16:27:50.425084 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 01 16:27:50.425084 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 16:27:50.425084 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 16:27:50.425084 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 16:27:50.425084 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 16:27:50.425084 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 16:27:50.425084 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 16:27:50.425084 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 16:27:50.425084 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application with Lock( Jul 01 16:27:50.425084 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 16:27:50.425084 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self._enter() Jul 01 16:27:50.425084 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 16:27:50.425084 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 16:27:50.425084 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 16:27:50.425084 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.creator() Jul 01 16:27:50.425084 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 16:27:50.425084 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application created_value = creator( Jul 01 16:27:50.425992 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 01 16:27:50.425992 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 01 16:27:50.425992 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 16:27:50.425992 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application Jul 01 16:27:50.428411 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 359/699] 10.222.0.57 () {56 vars in 1108 bytes} [Fri Jul 1 16:27:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 16:27:50.429563 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13bfa48-36ab-4bdc-bd5a-388c7deed4cc None admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:27:50.444028 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=100 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:27:50.454013 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225a8158-58d9-4375-bce6-16898cb94b03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:27:50.454766 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225a8158-58d9-4375-bce6-16898cb94b03 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:50.455051 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225a8158-58d9-4375-bce6-16898cb94b03 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:50.455328 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225a8158-58d9-4375-bce6-16898cb94b03 None admin] PATH_INFO: `/v3/domains` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:50.456058 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-225a8158-58d9-4375-bce6-16898cb94b03 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:27:50.456421 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-225a8158-58d9-4375-bce6-16898cb94b03 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:27:50.457809 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-225a8158-58d9-4375-bce6-16898cb94b03 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:27:50.477883 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 341/700] 10.222.0.57 () {56 vars in 1109 bytes} [Fri Jul 1 16:27:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:27:50.479607 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225a8158-58d9-4375-bce6-16898cb94b03 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:27:50.526560 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:27:50.551640 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13bfa48-36ab-4bdc-bd5a-388c7deed4cc None admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:27:50.598640 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225a8158-58d9-4375-bce6-16898cb94b03 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:27:50.605133 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7498ba76-8738-4961-988d-717a29de1d24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:27:50.605823 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7498ba76-8738-4961-988d-717a29de1d24 None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:50.606111 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7498ba76-8738-4961-988d-717a29de1d24 None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:50.606423 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7498ba76-8738-4961-988d-717a29de1d24 None admin] PATH_INFO: `/v3/domains` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:50.607174 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7498ba76-8738-4961-988d-717a29de1d24 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80312) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:27:50.607485 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7498ba76-8738-4961-988d-717a29de1d24 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:27:50.608871 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7498ba76-8738-4961-988d-717a29de1d24 None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:27:50.630378 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 360/701] 10.222.0.57 () {56 vars in 1109 bytes} [Fri Jul 1 16:27:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:27:50.630378 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7498ba76-8738-4961-988d-717a29de1d24 None admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:27:50.663169 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aed1603-3e4f-4e4e-9248-9501cab9f095 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:27:50.665223 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aed1603-3e4f-4e4e-9248-9501cab9f095 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:50.665870 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aed1603-3e4f-4e4e-9248-9501cab9f095 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:50.666460 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aed1603-3e4f-4e4e-9248-9501cab9f095 None admin] PATH_INFO: `/v3/domains` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:50.667618 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aed1603-3e4f-4e4e-9248-9501cab9f095 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:27:50.668088 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aed1603-3e4f-4e4e-9248-9501cab9f095 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:27:50.670243 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aed1603-3e4f-4e4e-9248-9501cab9f095 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:27:50.685577 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 342/702] 10.222.0.57 () {56 vars in 1109 bytes} [Fri Jul 1 16:27:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:27:50.687256 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aed1603-3e4f-4e4e-9248-9501cab9f095 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:27:50.726783 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=100 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:27:50.768401 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7498ba76-8738-4961-988d-717a29de1d24 None admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:27:50.809602 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3e3d73-6004-41cc-b7f8-854db114bcd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:27:50.810519 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3e3d73-6004-41cc-b7f8-854db114bcd1 None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:50.810808 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3e3d73-6004-41cc-b7f8-854db114bcd1 None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:50.811087 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3e3d73-6004-41cc-b7f8-854db114bcd1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:50.812659 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:27:50.817043 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aed1603-3e4f-4e4e-9248-9501cab9f095 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:27:50.818463 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea3e3d73-6004-41cc-b7f8-854db114bcd1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:27:50.820125 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea3e3d73-6004-41cc-b7f8-854db114bcd1 None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:27:50.837690 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application [None req-ea3e3d73-6004-41cc-b7f8-854db114bcd1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 01 16:27:50.837690 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 16:27:50.837690 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 16:27:50.837690 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 16:27:50.837690 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 16:27:50.837690 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 16:27:50.837690 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 16:27:50.837690 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 16:27:50.837690 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 16:27:50.837690 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 16:27:50.837690 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 16:27:50.837690 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 16:27:50.837690 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 01 16:27:50.837690 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 01 16:27:50.837690 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 01 16:27:50.838762 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 01 16:27:50.838762 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 16:27:50.838762 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 16:27:50.838762 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 16:27:50.838762 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 16:27:50.838762 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 16:27:50.838762 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 16:27:50.838762 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 16:27:50.838762 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application with Lock( Jul 01 16:27:50.838762 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 16:27:50.838762 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self._enter() Jul 01 16:27:50.838762 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 16:27:50.838762 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 16:27:50.838762 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 16:27:50.838762 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.creator() Jul 01 16:27:50.838762 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 16:27:50.838762 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application created_value = creator( Jul 01 16:27:50.839653 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 01 16:27:50.839653 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 01 16:27:50.839653 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 01 16:27:50.839653 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 01 16:27:50.839653 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 01 16:27:50.839653 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 01 16:27:50.839653 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 01 16:27:50.839653 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application Jul 01 16:27:50.840620 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 361/703] 10.222.0.57 () {56 vars in 1111 bytes} [Fri Jul 1 16:27:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 16:27:50.841293 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3e3d73-6004-41cc-b7f8-854db114bcd1 None admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:27:50.863600 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e92847-7906-4a71-97f5-704131b4f103 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:27:50.865142 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e92847-7906-4a71-97f5-704131b4f103 None admin] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:50.866609 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e92847-7906-4a71-97f5-704131b4f103 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:50.867347 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e92847-7906-4a71-97f5-704131b4f103 None admin] PATH_INFO: `/v3/users` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:50.869093 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e92847-7906-4a71-97f5-704131b4f103 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:27:50.871815 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e92847-7906-4a71-97f5-704131b4f103 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:27:50.900868 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 343/704] 10.222.0.57 () {60 vars in 1132 bytes} [Fri Jul 1 16:27:50 2022] POST /identity/v3/users => generated 254 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 16:27:50.903259 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748c9f8a-aca9-4d4e-afa7-3b5378b35c84 None None] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:50.903723 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748c9f8a-aca9-4d4e-afa7-3b5378b35c84 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:50.904099 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748c9f8a-aca9-4d4e-afa7-3b5378b35c84 None None] PATH_INFO: `/` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:50.910274 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 344/705] 10.222.0.57 () {54 vars in 926 bytes} [Fri Jul 1 16:27:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 16:27:50.910274 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748c9f8a-aca9-4d4e-afa7-3b5378b35c84 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:27:50.974227 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3e3d73-6004-41cc-b7f8-854db114bcd1 None admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:27:51.016198 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=100 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:27:51.025974 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41cb2a0b-a667-48c9-bc6d-e04bbfa08866 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:27:51.027073 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41cb2a0b-a667-48c9-bc6d-e04bbfa08866 None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:51.027507 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41cb2a0b-a667-48c9-bc6d-e04bbfa08866 None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:51.028731 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41cb2a0b-a667-48c9-bc6d-e04bbfa08866 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:51.040225 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41cb2a0b-a667-48c9-bc6d-e04bbfa08866 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:27:51.042272 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41cb2a0b-a667-48c9-bc6d-e04bbfa08866 None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:27:51.043658 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748c9f8a-aca9-4d4e-afa7-3b5378b35c84 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:27:51.062786 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application [None req-41cb2a0b-a667-48c9-bc6d-e04bbfa08866 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 01 16:27:51.062786 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 16:27:51.062786 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 16:27:51.062786 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 16:27:51.062786 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 16:27:51.062786 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 16:27:51.062786 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 16:27:51.062786 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 16:27:51.062786 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 16:27:51.062786 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 16:27:51.062786 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 16:27:51.062786 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 16:27:51.062786 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 01 16:27:51.062786 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 01 16:27:51.062786 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 01 16:27:51.063980 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 01 16:27:51.063980 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 16:27:51.063980 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 16:27:51.063980 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 16:27:51.063980 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 16:27:51.063980 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 16:27:51.063980 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 16:27:51.063980 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 16:27:51.063980 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application with Lock( Jul 01 16:27:51.063980 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 16:27:51.063980 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self._enter() Jul 01 16:27:51.063980 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 16:27:51.063980 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 16:27:51.063980 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 16:27:51.063980 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.creator() Jul 01 16:27:51.063980 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 16:27:51.063980 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application created_value = creator( Jul 01 16:27:51.065049 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 01 16:27:51.065049 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 01 16:27:51.065049 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 01 16:27:51.065049 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 01 16:27:51.065049 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 01 16:27:51.065049 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 01 16:27:51.065049 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 01 16:27:51.065049 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application Jul 01 16:27:51.068330 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 362/706] 10.222.0.57 () {56 vars in 1111 bytes} [Fri Jul 1 16:27:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 16:27:51.070914 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8797675-5e12-4fa9-9faa-567e39f396b3 None None] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:51.071430 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8797675-5e12-4fa9-9faa-567e39f396b3 None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:51.072341 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8797675-5e12-4fa9-9faa-567e39f396b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:51.093587 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b28076-a398-488a-96f2-d3803b846ebd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:27:51.094434 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b28076-a398-488a-96f2-d3803b846ebd None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:51.094820 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b28076-a398-488a-96f2-d3803b846ebd None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:51.095107 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.auth.core [None req-c8797675-5e12-4fa9-9faa-567e39f396b3 None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80312) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:27:51.095259 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b28076-a398-488a-96f2-d3803b846ebd None admin] PATH_INFO: `/v3/projects` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:51.096181 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b28076-a398-488a-96f2-d3803b846ebd None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:27:51.096502 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b28076-a398-488a-96f2-d3803b846ebd None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:27:51.096606 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-c8797675-5e12-4fa9-9faa-567e39f396b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:27:51.098892 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b28076-a398-488a-96f2-d3803b846ebd None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:27:51.115061 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 345/707] 10.222.0.57 () {56 vars in 1148 bytes} [Fri Jul 1 16:27:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:27:51.121919 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b28076-a398-488a-96f2-d3803b846ebd None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:27:51.144415 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:27:51.185160 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 363/708] 10.222.0.57 () {58 vars in 1023 bytes} [Fri Jul 1 16:27:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 01 16:27:51.186112 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8797675-5e12-4fa9-9faa-567e39f396b3 None None] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:27:51.255428 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b28076-a398-488a-96f2-d3803b846ebd None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:27:51.292410 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3dbfc96-07bc-4a1e-966d-230ced5f8b76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:27:51.293076 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3dbfc96-07bc-4a1e-966d-230ced5f8b76 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:51.293357 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3dbfc96-07bc-4a1e-966d-230ced5f8b76 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:51.293634 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3dbfc96-07bc-4a1e-966d-230ced5f8b76 None admin] PATH_INFO: `/v3/projects` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:51.294456 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3dbfc96-07bc-4a1e-966d-230ced5f8b76 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:27:51.295123 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3dbfc96-07bc-4a1e-966d-230ced5f8b76 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:27:51.296198 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3dbfc96-07bc-4a1e-966d-230ced5f8b76 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:27:51.307679 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8797675-5e12-4fa9-9faa-567e39f396b3 None None] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:27:51.309642 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 346/709] 10.222.0.57 () {56 vars in 1148 bytes} [Fri Jul 1 16:27:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:27:51.311425 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3dbfc96-07bc-4a1e-966d-230ced5f8b76 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:27:51.339042 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=100 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:27:51.346713 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96a071b-3f79-42ae-9ea9-e14b484468ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:27:51.347568 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96a071b-3f79-42ae-9ea9-e14b484468ef None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:51.348026 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96a071b-3f79-42ae-9ea9-e14b484468ef None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:51.348452 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96a071b-3f79-42ae-9ea9-e14b484468ef None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:51.349452 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96a071b-3f79-42ae-9ea9-e14b484468ef None admin] RBAC: Adding query filter params (role.id=e8809b67c9dc471daa1c2bd42c9a2baf, scope.project.id=e0788e6725a84e57a4174284c1845755, user.id=859fe410c2d2450fb803c6244839f27c) {{(pid=80312) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:27:51.350009 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96a071b-3f79-42ae-9ea9-e14b484468ef None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:27:51.351912 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96a071b-3f79-42ae-9ea9-e14b484468ef None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:27:51.359369 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 364/710] 10.222.0.57 () {56 vars in 1374 bytes} [Fri Jul 1 16:27:51 2022] GET /identity/v3/role_assignments?user.id=859fe410c2d2450fb803c6244839f27c&scope.project.id=e0788e6725a84e57a4174284c1845755&role.id=e8809b67c9dc471daa1c2bd42c9a2baf => generated 608 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:27:51.360580 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96a071b-3f79-42ae-9ea9-e14b484468ef None admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:27:51.403065 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:27:51.433929 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3dbfc96-07bc-4a1e-966d-230ced5f8b76 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:27:51.473716 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4733580d-b30a-4f5d-ad3b-c22dd202254b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:27:51.474832 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4733580d-b30a-4f5d-ad3b-c22dd202254b None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:51.475270 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4733580d-b30a-4f5d-ad3b-c22dd202254b None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:51.475615 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4733580d-b30a-4f5d-ad3b-c22dd202254b None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:51.476542 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4733580d-b30a-4f5d-ad3b-c22dd202254b None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:27:51.478230 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4733580d-b30a-4f5d-ad3b-c22dd202254b None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:27:51.480846 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96a071b-3f79-42ae-9ea9-e14b484468ef None admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:27:51.507229 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application [None req-4733580d-b30a-4f5d-ad3b-c22dd202254b None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 01 16:27:51.507229 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 16:27:51.507229 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 01 16:27:51.507229 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 01 16:27:51.507229 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 01 16:27:51.507229 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 01 16:27:51.507229 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 01 16:27:51.507229 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 01 16:27:51.507229 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 01 16:27:51.507229 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Jul 01 16:27:51.507229 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 01 16:27:51.507229 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Jul 01 16:27:51.507229 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 16:27:51.507229 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 16:27:51.507229 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 16:27:51.507229 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 16:27:51.508270 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 16:27:51.508270 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 16:27:51.508270 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 16:27:51.508270 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 16:27:51.508270 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 16:27:51.508270 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 16:27:51.508270 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 16:27:51.508270 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 01 16:27:51.508270 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 01 16:27:51.508270 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 01 16:27:51.508270 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 01 16:27:51.508270 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 16:27:51.508270 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 16:27:51.508270 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 16:27:51.508270 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 16:27:51.508270 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 16:27:51.509101 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 16:27:51.509101 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 16:27:51.509101 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application with Lock( Jul 01 16:27:51.509101 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 16:27:51.509101 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._enter() Jul 01 16:27:51.509101 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 16:27:51.509101 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 16:27:51.509101 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 16:27:51.509101 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.creator() Jul 01 16:27:51.509101 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 16:27:51.509101 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application created_value = creator( Jul 01 16:27:51.509101 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 01 16:27:51.509101 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 01 16:27:51.509101 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 01 16:27:51.509101 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Jul 01 16:27:51.509101 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 347/711] 10.222.0.57 () {56 vars in 1129 bytes} [Fri Jul 1 16:27:51 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 16:27:51.518147 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4733580d-b30a-4f5d-ad3b-c22dd202254b None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:27:51.531108 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7fe0eef-1783-43de-9bcd-0b18a04d2ed2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:27:51.532217 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fe0eef-1783-43de-9bcd-0b18a04d2ed2 None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:51.532753 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fe0eef-1783-43de-9bcd-0b18a04d2ed2 None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:51.533240 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fe0eef-1783-43de-9bcd-0b18a04d2ed2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:51.534471 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7fe0eef-1783-43de-9bcd-0b18a04d2ed2 None admin] RBAC: Adding query filter params (role.id=84ee739146934c1ab32c501ff8093644, scope.project.id=e0788e6725a84e57a4174284c1845755, user.id=7a50f745b31941b7a7265c0fc852f48e) {{(pid=80312) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:27:51.535073 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7fe0eef-1783-43de-9bcd-0b18a04d2ed2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:27:51.537037 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7fe0eef-1783-43de-9bcd-0b18a04d2ed2 None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:27:51.544642 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 365/712] 10.222.0.57 () {56 vars in 1374 bytes} [Fri Jul 1 16:27:51 2022] GET /identity/v3/role_assignments?user.id=7a50f745b31941b7a7265c0fc852f48e&scope.project.id=e0788e6725a84e57a4174284c1845755&role.id=84ee739146934c1ab32c501ff8093644 => generated 259 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:27:51.629131 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0bd53e-2cfe-4013-962a-f5926297ef6d None None] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:51.629785 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0bd53e-2cfe-4013-962a-f5926297ef6d None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:51.630261 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0bd53e-2cfe-4013-962a-f5926297ef6d None None] PATH_INFO: `/` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:51.632591 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 366/713] 10.222.0.57 () {54 vars in 926 bytes} [Fri Jul 1 16:27:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 16:27:51.639614 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6715a04-98fb-45a6-9263-d0d8887ef10b None None] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:51.640202 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6715a04-98fb-45a6-9263-d0d8887ef10b None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:51.640670 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6715a04-98fb-45a6-9263-d0d8887ef10b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:51.675269 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.auth.core [None req-d6715a04-98fb-45a6-9263-d0d8887ef10b None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80312) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:27:51.677889 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-d6715a04-98fb-45a6-9263-d0d8887ef10b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:27:51.681946 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4733580d-b30a-4f5d-ad3b-c22dd202254b None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:27:51.726538 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476bd7fe-fbc5-4257-8987-01f347f9b6f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:27:51.727214 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476bd7fe-fbc5-4257-8987-01f347f9b6f3 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:51.727504 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476bd7fe-fbc5-4257-8987-01f347f9b6f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:51.727786 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476bd7fe-fbc5-4257-8987-01f347f9b6f3 None admin] PATH_INFO: `/v3/services` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:51.728518 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476bd7fe-fbc5-4257-8987-01f347f9b6f3 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:27:51.728831 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476bd7fe-fbc5-4257-8987-01f347f9b6f3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:27:51.730229 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476bd7fe-fbc5-4257-8987-01f347f9b6f3 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:27:51.735804 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 348/714] 10.222.0.57 () {56 vars in 1124 bytes} [Fri Jul 1 16:27:51 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 01 16:27:51.741574 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=100 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:27:51.741752 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476bd7fe-fbc5-4257-8987-01f347f9b6f3 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:27:51.745529 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:27:51.772927 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 367/715] 10.222.0.57 () {58 vars in 1023 bytes} [Fri Jul 1 16:27:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 01 16:27:51.803823 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6715a04-98fb-45a6-9263-d0d8887ef10b None None] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:27:51.876085 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476bd7fe-fbc5-4257-8987-01f347f9b6f3 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:27:51.915646 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7d6dd1-21b2-434e-8158-0fa34fec66ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:27:51.916334 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7d6dd1-21b2-434e-8158-0fa34fec66ec None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:51.916681 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7d6dd1-21b2-434e-8158-0fa34fec66ec None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:51.916965 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7d6dd1-21b2-434e-8158-0fa34fec66ec None admin] PATH_INFO: `/v3/services` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:51.917734 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7d6dd1-21b2-434e-8158-0fa34fec66ec None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:27:51.918160 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7d6dd1-21b2-434e-8158-0fa34fec66ec None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:27:51.919686 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7d6dd1-21b2-434e-8158-0fa34fec66ec None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:27:51.929614 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 349/716] 10.222.0.57 () {56 vars in 1124 bytes} [Fri Jul 1 16:27:51 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 01 16:27:51.945650 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6715a04-98fb-45a6-9263-d0d8887ef10b None None] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:27:52.004470 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f893d4-59ce-491a-ad2f-19b979866dd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:27:52.005296 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f893d4-59ce-491a-ad2f-19b979866dd2 None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:52.005584 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f893d4-59ce-491a-ad2f-19b979866dd2 None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:52.005863 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f893d4-59ce-491a-ad2f-19b979866dd2 None admin] PATH_INFO: `/v3/services/network` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:52.006611 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f893d4-59ce-491a-ad2f-19b979866dd2 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:27:52.012048 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f893d4-59ce-491a-ad2f-19b979866dd2 None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:27:52.035779 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application [None req-07f893d4-59ce-491a-ad2f-19b979866dd2 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 01 16:27:52.035779 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 16:27:52.035779 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 01 16:27:52.035779 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 01 16:27:52.035779 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 01 16:27:52.035779 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 01 16:27:52.035779 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 01 16:27:52.035779 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 01 16:27:52.035779 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 01 16:27:52.035779 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application Jul 01 16:27:52.035779 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 01 16:27:52.035779 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application Jul 01 16:27:52.035779 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 16:27:52.035779 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 16:27:52.035779 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 16:27:52.035779 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 16:27:52.035779 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 16:27:52.037177 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 16:27:52.037177 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 16:27:52.037177 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 16:27:52.037177 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 16:27:52.037177 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 16:27:52.037177 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 16:27:52.037177 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 01 16:27:52.037177 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 01 16:27:52.037177 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 01 16:27:52.037177 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 01 16:27:52.037177 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 16:27:52.037177 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 16:27:52.037177 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 16:27:52.037177 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 16:27:52.037177 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 16:27:52.037177 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 16:27:52.038958 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 16:27:52.038958 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application with Lock( Jul 01 16:27:52.038958 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 16:27:52.038958 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self._enter() Jul 01 16:27:52.038958 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 16:27:52.038958 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 16:27:52.038958 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 16:27:52.038958 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.creator() Jul 01 16:27:52.038958 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 16:27:52.038958 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application created_value = creator( Jul 01 16:27:52.038958 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 01 16:27:52.038958 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 01 16:27:52.038958 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 01 16:27:52.038958 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application Jul 01 16:27:52.038958 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 368/717] 10.222.0.57 () {56 vars in 1111 bytes} [Fri Jul 1 16:27:51 2022] GET /identity/v3/services/network => generated 88 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 01 16:27:52.043898 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7d6dd1-21b2-434e-8158-0fa34fec66ec None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:27:52.165926 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:27:52.208280 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7d6dd1-21b2-434e-8158-0fa34fec66ec None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:27:52.245365 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808749c8-176f-4b98-aefe-63ece16b2ec6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:27:52.246061 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808749c8-176f-4b98-aefe-63ece16b2ec6 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:52.246445 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808749c8-176f-4b98-aefe-63ece16b2ec6 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:52.246790 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808749c8-176f-4b98-aefe-63ece16b2ec6 None admin] PATH_INFO: `/v3/services` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:52.247688 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-808749c8-176f-4b98-aefe-63ece16b2ec6 None admin] RBAC: Adding query filter params (name=network) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:27:52.248050 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-808749c8-176f-4b98-aefe-63ece16b2ec6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:27:52.249784 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-808749c8-176f-4b98-aefe-63ece16b2ec6 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:27:52.257176 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 350/718] 10.222.0.57 () {56 vars in 1112 bytes} [Fri Jul 1 16:27:52 2022] GET /identity/v3/services?name=network => generated 124 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:27:52.262525 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f893d4-59ce-491a-ad2f-19b979866dd2 None admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:27:52.329351 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=100 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:27:52.417274 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f893d4-59ce-491a-ad2f-19b979866dd2 None admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:27:52.462392 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25312f47-d653-4a84-88ad-81b2465ff2f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:27:52.463070 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25312f47-d653-4a84-88ad-81b2465ff2f9 None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:52.463353 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25312f47-d653-4a84-88ad-81b2465ff2f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:52.463630 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25312f47-d653-4a84-88ad-81b2465ff2f9 None admin] PATH_INFO: `/v3/services` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:52.464723 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25312f47-d653-4a84-88ad-81b2465ff2f9 None admin] RBAC: Adding query filter params (type=network) {{(pid=80312) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:27:52.465195 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25312f47-d653-4a84-88ad-81b2465ff2f9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:27:52.467445 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25312f47-d653-4a84-88ad-81b2465ff2f9 None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:27:52.478271 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 369/719] 10.222.0.57 () {56 vars in 1112 bytes} [Fri Jul 1 16:27:52 2022] GET /identity/v3/services?type=network => generated 124 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:27:53.484289 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594fc321-84f1-4626-a004-ad3c81eba0fe None None] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:53.484819 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594fc321-84f1-4626-a004-ad3c81eba0fe None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:53.484928 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594fc321-84f1-4626-a004-ad3c81eba0fe None None] PATH_INFO: `/` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:53.486657 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 351/720] 10.222.0.57 () {54 vars in 926 bytes} [Fri Jul 1 16:27:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 16:27:53.492525 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f05aeab-7b7b-4f8a-a85e-4c562d38ff66 None None] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:53.493083 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f05aeab-7b7b-4f8a-a85e-4c562d38ff66 None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:53.493427 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f05aeab-7b7b-4f8a-a85e-4c562d38ff66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:53.515943 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.auth.core [None req-9f05aeab-7b7b-4f8a-a85e-4c562d38ff66 None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80312) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:27:53.517152 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-9f05aeab-7b7b-4f8a-a85e-4c562d38ff66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:27:53.615068 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 370/721] 10.222.0.57 () {58 vars in 1023 bytes} [Fri Jul 1 16:27:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 01 16:27:53.672080 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594fc321-84f1-4626-a004-ad3c81eba0fe None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:27:53.821567 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594fc321-84f1-4626-a004-ad3c81eba0fe None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:27:53.864603 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:27:53.872808 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceeecbc4-c787-4e4f-896b-556a01286a74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:27:53.873923 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceeecbc4-c787-4e4f-896b-556a01286a74 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:53.874450 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceeecbc4-c787-4e4f-896b-556a01286a74 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:53.874959 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceeecbc4-c787-4e4f-896b-556a01286a74 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:53.891475 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceeecbc4-c787-4e4f-896b-556a01286a74 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 01 16:27:53.900259 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application [None req-ceeecbc4-c787-4e4f-896b-556a01286a74 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 01 16:27:53.900259 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 16:27:53.900259 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 16:27:53.900259 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 16:27:53.900259 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 16:27:53.900259 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 16:27:53.900259 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 16:27:53.900259 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 16:27:53.900259 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 16:27:53.900259 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 16:27:53.900259 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 16:27:53.900259 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 16:27:53.900259 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 01 16:27:53.900259 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 01 16:27:53.900259 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 01 16:27:53.900259 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 01 16:27:53.901275 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 01 16:27:53.901275 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 01 16:27:53.901275 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 01 16:27:53.901275 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 01 16:27:53.901275 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 16:27:53.901275 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 16:27:53.901275 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 16:27:53.901275 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 16:27:53.901275 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 16:27:53.901275 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 16:27:53.901275 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 16:27:53.901275 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application with Lock( Jul 01 16:27:53.901275 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 16:27:53.901275 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._enter() Jul 01 16:27:53.901275 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 16:27:53.901275 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 16:27:53.902142 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 16:27:53.902142 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.creator() Jul 01 16:27:53.902142 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 16:27:53.902142 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application created_value = creator( Jul 01 16:27:53.902142 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 01 16:27:53.902142 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 01 16:27:53.902142 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 01 16:27:53.902142 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 01 16:27:53.902142 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 01 16:27:53.902142 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 01 16:27:53.902142 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 01 16:27:53.902142 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Jul 01 16:27:53.904902 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 352/722] 10.222.0.57 () {56 vars in 1102 bytes} [Fri Jul 1 16:27:53 2022] GET /identity/v3/roles/service => generated 85 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 16:27:53.916157 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f05aeab-7b7b-4f8a-a85e-4c562d38ff66 None None] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:27:53.977310 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=100 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:27:54.069284 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f05aeab-7b7b-4f8a-a85e-4c562d38ff66 None None] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:27:54.109939 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8932ebd5-f976-4d73-b48d-bead53a059ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:27:54.110568 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8932ebd5-f976-4d73-b48d-bead53a059ed None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:54.110859 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8932ebd5-f976-4d73-b48d-bead53a059ed None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:54.111144 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8932ebd5-f976-4d73-b48d-bead53a059ed None admin] PATH_INFO: `/v3/roles` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:54.111911 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8932ebd5-f976-4d73-b48d-bead53a059ed None admin] RBAC: Adding query filter params (name=service) {{(pid=80312) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:27:54.112221 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8932ebd5-f976-4d73-b48d-bead53a059ed None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:27:54.113599 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8932ebd5-f976-4d73-b48d-bead53a059ed None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:27:54.122976 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 371/723] 10.222.0.57 () {56 vars in 1103 bytes} [Fri Jul 1 16:27:53 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:27:54.131481 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceeecbc4-c787-4e4f-896b-556a01286a74 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:27:54.265931 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824424ae-60d2-49e8-8fc5-3a5bac2d6923 None None] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:54.266510 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824424ae-60d2-49e8-8fc5-3a5bac2d6923 None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:54.266930 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824424ae-60d2-49e8-8fc5-3a5bac2d6923 None None] PATH_INFO: `/` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:54.268695 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 372/724] 10.222.0.57 () {54 vars in 926 bytes} [Fri Jul 1 16:27:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 16:27:54.275499 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf123bcd-75d0-4e89-99a9-924641d15571 None None] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:54.276100 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf123bcd-75d0-4e89-99a9-924641d15571 None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:54.276640 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf123bcd-75d0-4e89-99a9-924641d15571 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:54.288229 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceeecbc4-c787-4e4f-896b-556a01286a74 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:27:54.311846 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.auth.core [None req-cf123bcd-75d0-4e89-99a9-924641d15571 None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80312) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:27:54.313261 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-cf123bcd-75d0-4e89-99a9-924641d15571 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:27:54.375155 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2893d685-91f8-4b6b-8c6c-169e03f544a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:27:54.375883 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2893d685-91f8-4b6b-8c6c-169e03f544a2 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:54.376170 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2893d685-91f8-4b6b-8c6c-169e03f544a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:54.376454 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2893d685-91f8-4b6b-8c6c-169e03f544a2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:54.387868 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2893d685-91f8-4b6b-8c6c-169e03f544a2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:27:54.389334 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2893d685-91f8-4b6b-8c6c-169e03f544a2 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:27:54.390792 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 373/725] 10.222.0.57 () {58 vars in 1023 bytes} [Fri Jul 1 16:27:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 01 16:27:54.392652 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8587cb-c6f7-4616-a95c-930a8bb11036 None None] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:54.392948 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8587cb-c6f7-4616-a95c-930a8bb11036 None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:54.393230 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8587cb-c6f7-4616-a95c-930a8bb11036 None None] PATH_INFO: `/` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:54.394775 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 374/726] 10.222.0.57 () {54 vars in 926 bytes} [Fri Jul 1 16:27:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 16:27:54.402398 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a5ebcd-b3fe-4e8e-834c-25377cbe0e98 None None] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:54.403257 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a5ebcd-b3fe-4e8e-834c-25377cbe0e98 None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:54.403641 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a5ebcd-b3fe-4e8e-834c-25377cbe0e98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:54.413604 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application [None req-2893d685-91f8-4b6b-8c6c-169e03f544a2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 16:27:54.413604 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 16:27:54.413604 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 16:27:54.413604 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 16:27:54.413604 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 16:27:54.413604 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 16:27:54.413604 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 16:27:54.413604 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 16:27:54.413604 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 16:27:54.413604 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 16:27:54.413604 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 16:27:54.413604 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 16:27:54.413604 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 01 16:27:54.413604 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 01 16:27:54.413604 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 01 16:27:54.414575 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 01 16:27:54.414575 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 16:27:54.414575 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 16:27:54.414575 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 16:27:54.414575 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 16:27:54.414575 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 16:27:54.414575 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 16:27:54.414575 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 16:27:54.414575 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application with Lock( Jul 01 16:27:54.414575 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 16:27:54.414575 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._enter() Jul 01 16:27:54.414575 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 16:27:54.414575 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 16:27:54.414575 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 16:27:54.414575 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.creator() Jul 01 16:27:54.414575 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 16:27:54.414575 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application created_value = creator( Jul 01 16:27:54.415958 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 01 16:27:54.415958 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 01 16:27:54.415958 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 16:27:54.415958 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Jul 01 16:27:54.415958 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 353/727] 10.222.0.57 () {56 vars in 1108 bytes} [Fri Jul 1 16:27:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 284 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 16:27:54.420662 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2893d685-91f8-4b6b-8c6c-169e03f544a2 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:27:54.436594 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.auth.core [None req-93a5ebcd-b3fe-4e8e-834c-25377cbe0e98 None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80312) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:27:54.438184 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-93a5ebcd-b3fe-4e8e-834c-25377cbe0e98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:27:54.468775 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=100 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:27:54.480812 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:27:54.525395 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 375/728] 10.222.0.57 () {58 vars in 1023 bytes} [Fri Jul 1 16:27:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 01 16:27:54.526928 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a5ebcd-b3fe-4e8e-834c-25377cbe0e98 None None] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:27:54.575691 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2893d685-91f8-4b6b-8c6c-169e03f544a2 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:27:54.613167 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f58bac-da53-4932-b861-19286e3bbc4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:27:54.613993 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f58bac-da53-4932-b861-19286e3bbc4d None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:54.614418 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f58bac-da53-4932-b861-19286e3bbc4d None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:54.614802 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f58bac-da53-4932-b861-19286e3bbc4d None admin] PATH_INFO: `/v3/domains` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:54.615848 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f58bac-da53-4932-b861-19286e3bbc4d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:27:54.616264 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f58bac-da53-4932-b861-19286e3bbc4d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:27:54.617868 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f58bac-da53-4932-b861-19286e3bbc4d None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:27:54.633898 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 354/729] 10.222.0.57 () {56 vars in 1109 bytes} [Fri Jul 1 16:27:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 01 16:27:54.634869 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f58bac-da53-4932-b861-19286e3bbc4d None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:27:54.678980 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a5ebcd-b3fe-4e8e-834c-25377cbe0e98 None None] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:27:54.718540 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3017f988-94b0-4b7a-803f-692deefe7110 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:27:54.719443 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3017f988-94b0-4b7a-803f-692deefe7110 None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:54.719736 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3017f988-94b0-4b7a-803f-692deefe7110 None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:54.720013 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3017f988-94b0-4b7a-803f-692deefe7110 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:54.732248 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3017f988-94b0-4b7a-803f-692deefe7110 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 01 16:27:54.741817 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application [None req-3017f988-94b0-4b7a-803f-692deefe7110 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 01 16:27:54.741817 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 16:27:54.741817 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 16:27:54.741817 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 16:27:54.741817 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 16:27:54.741817 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 16:27:54.741817 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 16:27:54.741817 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 16:27:54.741817 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 16:27:54.741817 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 16:27:54.741817 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 16:27:54.741817 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 16:27:54.741817 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 01 16:27:54.741817 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 01 16:27:54.741817 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 01 16:27:54.741817 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 01 16:27:54.744082 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 01 16:27:54.744082 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 01 16:27:54.744082 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 01 16:27:54.744082 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 01 16:27:54.744082 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 16:27:54.744082 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 16:27:54.744082 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 16:27:54.744082 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 16:27:54.744082 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 16:27:54.744082 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 16:27:54.744082 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 16:27:54.744082 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application with Lock( Jul 01 16:27:54.744082 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 16:27:54.744082 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self._enter() Jul 01 16:27:54.744082 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 16:27:54.744082 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 16:27:54.746061 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 16:27:54.746061 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.creator() Jul 01 16:27:54.746061 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 16:27:54.746061 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application created_value = creator( Jul 01 16:27:54.746061 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 01 16:27:54.746061 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 01 16:27:54.746061 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 01 16:27:54.746061 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 01 16:27:54.746061 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 01 16:27:54.746061 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 01 16:27:54.746061 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 01 16:27:54.746061 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application Jul 01 16:27:54.746061 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 376/730] 10.222.0.57 () {56 vars in 1096 bytes} [Fri Jul 1 16:27:54 2022] GET /identity/v3/roles/admin => generated 83 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 16:27:54.746061 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3017f988-94b0-4b7a-803f-692deefe7110 None admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:27:54.747078 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f58bac-da53-4932-b861-19286e3bbc4d None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:27:54.747078 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:27:54.767644 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=100 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:27:54.784253 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ef1b73-d0b1-432a-a298-b2be6070d952 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:27:54.785205 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ef1b73-d0b1-432a-a298-b2be6070d952 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:54.785594 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ef1b73-d0b1-432a-a298-b2be6070d952 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:54.786006 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ef1b73-d0b1-432a-a298-b2be6070d952 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:54.797765 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ef1b73-d0b1-432a-a298-b2be6070d952 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 01 16:27:54.807227 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application [None req-47ef1b73-d0b1-432a-a298-b2be6070d952 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 01 16:27:54.807227 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 16:27:54.807227 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 16:27:54.807227 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 16:27:54.807227 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 16:27:54.807227 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 16:27:54.807227 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 16:27:54.807227 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 16:27:54.807227 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 16:27:54.807227 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 16:27:54.807227 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 16:27:54.807227 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 16:27:54.807227 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 01 16:27:54.807227 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 01 16:27:54.807227 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 01 16:27:54.807227 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 01 16:27:54.809099 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 01 16:27:54.809099 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 01 16:27:54.809099 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 01 16:27:54.809099 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 01 16:27:54.809099 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 16:27:54.809099 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 16:27:54.809099 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 16:27:54.809099 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 16:27:54.809099 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 16:27:54.809099 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 16:27:54.809099 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 16:27:54.809099 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application with Lock( Jul 01 16:27:54.809099 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 16:27:54.809099 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._enter() Jul 01 16:27:54.809099 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 16:27:54.809099 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 16:27:54.811218 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 16:27:54.811218 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.creator() Jul 01 16:27:54.811218 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 16:27:54.811218 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application created_value = creator( Jul 01 16:27:54.811218 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 01 16:27:54.811218 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 01 16:27:54.811218 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 01 16:27:54.811218 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 01 16:27:54.811218 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 01 16:27:54.811218 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 01 16:27:54.811218 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 01 16:27:54.811218 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Jul 01 16:27:54.811218 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 355/731] 10.222.0.57 () {56 vars in 1096 bytes} [Fri Jul 1 16:27:54 2022] GET /identity/v3/roles/admin => generated 83 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 16:27:54.812903 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ef1b73-d0b1-432a-a298-b2be6070d952 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:27:54.879053 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3017f988-94b0-4b7a-803f-692deefe7110 None admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:27:54.920437 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62559432-2e09-4ba9-89b2-a30b94960f42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:27:54.921587 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62559432-2e09-4ba9-89b2-a30b94960f42 None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:54.922039 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62559432-2e09-4ba9-89b2-a30b94960f42 None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:54.922721 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62559432-2e09-4ba9-89b2-a30b94960f42 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:54.924264 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ef1b73-d0b1-432a-a298-b2be6070d952 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:27:54.933688 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62559432-2e09-4ba9-89b2-a30b94960f42 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:27:54.935525 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62559432-2e09-4ba9-89b2-a30b94960f42 None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:27:54.952310 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application [None req-62559432-2e09-4ba9-89b2-a30b94960f42 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 01 16:27:54.952310 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 16:27:54.952310 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 16:27:54.952310 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 16:27:54.952310 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 16:27:54.952310 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 16:27:54.952310 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 16:27:54.952310 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 16:27:54.952310 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 16:27:54.952310 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 16:27:54.952310 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 16:27:54.952310 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 16:27:54.952310 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 01 16:27:54.952310 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 01 16:27:54.952310 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 01 16:27:54.952310 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 01 16:27:54.953730 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 16:27:54.953730 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 16:27:54.953730 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 01 16:27:54.953730 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 01 16:27:54.953730 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 01 16:27:54.953730 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 01 16:27:54.953730 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 16:27:54.953730 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 16:27:54.953730 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 16:27:54.953730 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 16:27:54.953730 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 16:27:54.953730 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application with Lock( Jul 01 16:27:54.953730 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 16:27:54.953730 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self._enter() Jul 01 16:27:54.953730 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 16:27:54.953730 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 16:27:54.953730 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 16:27:54.953730 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.creator() Jul 01 16:27:54.955556 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 16:27:54.955556 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application created_value = creator( Jul 01 16:27:54.955556 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 01 16:27:54.955556 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 01 16:27:54.955556 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 01 16:27:54.955556 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 01 16:27:54.955556 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 01 16:27:54.955556 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 01 16:27:54.955556 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 01 16:27:54.955556 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application Jul 01 16:27:54.955556 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 377/732] 10.222.0.57 () {56 vars in 1117 bytes} [Fri Jul 1 16:27:54 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 16:27:54.955556 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62559432-2e09-4ba9-89b2-a30b94960f42 None admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:27:54.969637 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b22830-dd38-46c3-95eb-dfe9c01114b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:27:54.970311 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b22830-dd38-46c3-95eb-dfe9c01114b3 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:54.970730 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b22830-dd38-46c3-95eb-dfe9c01114b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:54.971008 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b22830-dd38-46c3-95eb-dfe9c01114b3 None admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:54.971765 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b22830-dd38-46c3-95eb-dfe9c01114b3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:27:54.972074 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b22830-dd38-46c3-95eb-dfe9c01114b3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:27:54.973512 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b22830-dd38-46c3-95eb-dfe9c01114b3 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:27:54.985421 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 356/733] 10.222.0.57 () {56 vars in 1099 bytes} [Fri Jul 1 16:27:54 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:27:54.986795 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac67a1b4-5b9c-42ad-a338-532081df4b2e None None] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:54.987499 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac67a1b4-5b9c-42ad-a338-532081df4b2e None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:54.987499 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac67a1b4-5b9c-42ad-a338-532081df4b2e None None] PATH_INFO: `/` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:54.990026 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 357/734] 10.222.0.57 () {54 vars in 926 bytes} [Fri Jul 1 16:27:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 16:27:54.990843 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0019cc0-d556-4de1-b393-73d1c8a8a175 None None] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:54.991259 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0019cc0-d556-4de1-b393-73d1c8a8a175 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:54.991692 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0019cc0-d556-4de1-b393-73d1c8a8a175 None None] PATH_INFO: `/` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:54.993477 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 358/735] 10.222.0.57 () {54 vars in 926 bytes} [Fri Jul 1 16:27:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 16:27:54.994219 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0019cc0-d556-4de1-b393-73d1c8a8a175 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:27:55.021264 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:27:55.046446 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=100 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:27:55.068883 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62559432-2e09-4ba9-89b2-a30b94960f42 None admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:27:55.103880 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3c1022-2792-4397-8ca5-998412c0fa04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:27:55.104552 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3c1022-2792-4397-8ca5-998412c0fa04 None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:55.104836 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3c1022-2792-4397-8ca5-998412c0fa04 None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:55.104943 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0019cc0-d556-4de1-b393-73d1c8a8a175 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:27:55.105113 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3c1022-2792-4397-8ca5-998412c0fa04 None admin] PATH_INFO: `/v3/roles` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:55.105864 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d3c1022-2792-4397-8ca5-998412c0fa04 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80312) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:27:55.106175 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d3c1022-2792-4397-8ca5-998412c0fa04 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:27:55.107587 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d3c1022-2792-4397-8ca5-998412c0fa04 None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:27:55.117010 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 378/736] 10.222.0.57 () {56 vars in 1099 bytes} [Fri Jul 1 16:27:54 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:27:55.117863 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3c1022-2792-4397-8ca5-998412c0fa04 None admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:27:55.143783 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f7d2c7-7f04-41fc-827f-53c76578529b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:27:55.144450 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f7d2c7-7f04-41fc-827f-53c76578529b None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:55.144738 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f7d2c7-7f04-41fc-827f-53c76578529b None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:55.145013 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f7d2c7-7f04-41fc-827f-53c76578529b None admin] PATH_INFO: `/v3/users` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:55.145844 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f7d2c7-7f04-41fc-827f-53c76578529b None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:27:55.146155 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f7d2c7-7f04-41fc-827f-53c76578529b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:27:55.148206 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f7d2c7-7f04-41fc-827f-53c76578529b None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:27:55.165295 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 359/737] 10.222.0.57 () {56 vars in 1149 bytes} [Fri Jul 1 16:27:54 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:27:55.166546 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cd101a-9b8c-4f87-8c71-c75e8971b158 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:55.166835 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cd101a-9b8c-4f87-8c71-c75e8971b158 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:55.167114 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cd101a-9b8c-4f87-8c71-c75e8971b158 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:55.183024 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-c2cd101a-9b8c-4f87-8c71-c75e8971b158 None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:27:55.184153 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-c2cd101a-9b8c-4f87-8c71-c75e8971b158 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:27:55.222426 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3c1022-2792-4397-8ca5-998412c0fa04 None admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:27:55.249961 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 360/738] 10.222.0.57 () {58 vars in 1023 bytes} [Fri Jul 1 16:27:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 01 16:27:55.251277 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb116df-322a-413d-98dc-3261481d5ac2 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:55.251569 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb116df-322a-413d-98dc-3261481d5ac2 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:55.251850 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb116df-322a-413d-98dc-3261481d5ac2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:55.266599 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c47ec77-95ae-4347-a479-b53d6758aaea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:27:55.268003 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c47ec77-95ae-4347-a479-b53d6758aaea None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:55.268301 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c47ec77-95ae-4347-a479-b53d6758aaea None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:55.268585 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c47ec77-95ae-4347-a479-b53d6758aaea None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:55.276173 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-dcb116df-322a-413d-98dc-3261481d5ac2 None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:27:55.278257 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-dcb116df-322a-413d-98dc-3261481d5ac2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:27:55.281011 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=100 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:27:55.285935 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c47ec77-95ae-4347-a479-b53d6758aaea None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:27:55.288004 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c47ec77-95ae-4347-a479-b53d6758aaea None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:27:55.292373 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:27:55.306760 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application [None req-6c47ec77-95ae-4347-a479-b53d6758aaea None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 16:27:55.306760 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 16:27:55.306760 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 16:27:55.306760 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 16:27:55.306760 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 16:27:55.306760 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 16:27:55.306760 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 16:27:55.306760 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 16:27:55.306760 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 16:27:55.306760 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 16:27:55.306760 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 16:27:55.306760 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 16:27:55.306760 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 01 16:27:55.306760 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 01 16:27:55.306760 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 01 16:27:55.307804 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 01 16:27:55.307804 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 16:27:55.307804 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 16:27:55.307804 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 16:27:55.307804 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 16:27:55.307804 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 16:27:55.307804 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 16:27:55.307804 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 16:27:55.307804 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application with Lock( Jul 01 16:27:55.307804 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 16:27:55.307804 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self._enter() Jul 01 16:27:55.307804 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 16:27:55.307804 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 16:27:55.307804 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 16:27:55.307804 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.creator() Jul 01 16:27:55.307804 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 16:27:55.307804 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application created_value = creator( Jul 01 16:27:55.308720 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 01 16:27:55.308720 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 01 16:27:55.308720 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 16:27:55.308720 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application Jul 01 16:27:55.308720 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 379/739] 10.222.0.57 () {56 vars in 1108 bytes} [Fri Jul 1 16:27:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 16:27:55.308720 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c47ec77-95ae-4347-a479-b53d6758aaea None admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:27:55.354933 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 361/740] 10.222.0.57 () {58 vars in 1023 bytes} [Fri Jul 1 16:27:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 01 16:27:55.355764 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb116df-322a-413d-98dc-3261481d5ac2 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:27:55.420112 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c47ec77-95ae-4347-a479-b53d6758aaea None admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:27:55.458671 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256ebdc7-16e2-4ace-bc35-6f2ca152aefe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:27:55.459386 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256ebdc7-16e2-4ace-bc35-6f2ca152aefe None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:55.460011 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256ebdc7-16e2-4ace-bc35-6f2ca152aefe None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:55.460011 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256ebdc7-16e2-4ace-bc35-6f2ca152aefe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:55.470427 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-256ebdc7-16e2-4ace-bc35-6f2ca152aefe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:27:55.471875 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-256ebdc7-16e2-4ace-bc35-6f2ca152aefe None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:27:55.474286 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb116df-322a-413d-98dc-3261481d5ac2 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:27:55.488717 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application [None req-256ebdc7-16e2-4ace-bc35-6f2ca152aefe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 16:27:55.488717 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 16:27:55.488717 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 16:27:55.488717 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 16:27:55.488717 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 16:27:55.488717 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 16:27:55.488717 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 16:27:55.488717 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 16:27:55.488717 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 16:27:55.488717 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 16:27:55.488717 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 16:27:55.488717 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 16:27:55.488717 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 01 16:27:55.488717 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 01 16:27:55.488717 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 01 16:27:55.489735 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 01 16:27:55.489735 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 16:27:55.489735 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 16:27:55.489735 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 16:27:55.489735 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 16:27:55.489735 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 16:27:55.489735 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 16:27:55.489735 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 16:27:55.489735 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application with Lock( Jul 01 16:27:55.489735 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 16:27:55.489735 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self._enter() Jul 01 16:27:55.489735 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 16:27:55.489735 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 16:27:55.489735 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 16:27:55.489735 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.creator() Jul 01 16:27:55.489735 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 16:27:55.489735 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application created_value = creator( Jul 01 16:27:55.491213 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 01 16:27:55.491213 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 01 16:27:55.491213 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 16:27:55.491213 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application Jul 01 16:27:55.491213 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 380/741] 10.222.0.57 () {56 vars in 1108 bytes} [Fri Jul 1 16:27:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 16:27:55.491632 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256ebdc7-16e2-4ace-bc35-6f2ca152aefe None admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:27:55.517662 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b45236-327e-445b-bf68-66f6cf3c7c63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:27:55.518318 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b45236-327e-445b-bf68-66f6cf3c7c63 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:55.518636 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b45236-327e-445b-bf68-66f6cf3c7c63 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:55.518942 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b45236-327e-445b-bf68-66f6cf3c7c63 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:55.530988 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:27:55.538144 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=100 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:27:55.595883 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b45236-327e-445b-bf68-66f6cf3c7c63 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:27:55.597331 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b45236-327e-445b-bf68-66f6cf3c7c63 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:27:55.597469 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256ebdc7-16e2-4ace-bc35-6f2ca152aefe None admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:27:55.637788 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b572e4e4-6ea7-4dd3-9cfb-6e4a916cd8d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:27:55.638475 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b572e4e4-6ea7-4dd3-9cfb-6e4a916cd8d7 None admin] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:55.638758 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b572e4e4-6ea7-4dd3-9cfb-6e4a916cd8d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:55.639039 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b572e4e4-6ea7-4dd3-9cfb-6e4a916cd8d7 None admin] PATH_INFO: `/v3/services` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:55.639758 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b572e4e4-6ea7-4dd3-9cfb-6e4a916cd8d7 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:27:55.641124 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b572e4e4-6ea7-4dd3-9cfb-6e4a916cd8d7 None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:27:55.656594 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 381/742] 10.222.0.57 () {60 vars in 1141 bytes} [Fri Jul 1 16:27:55 2022] POST /identity/v3/services => generated 251 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 16:27:55.657424 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b572e4e4-6ea7-4dd3-9cfb-6e4a916cd8d7 None admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:27:55.715867 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 362/743] 10.222.0.57 () {58 vars in 1282 bytes} [Fri Jul 1 16:27:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 360 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 01 16:27:55.717024 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b45236-327e-445b-bf68-66f6cf3c7c63 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:27:55.770996 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:27:55.820217 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:27:55.827206 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b572e4e4-6ea7-4dd3-9cfb-6e4a916cd8d7 None admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:27:55.856148 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b45236-327e-445b-bf68-66f6cf3c7c63 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:27:55.880145 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27349fcc-2aef-4ecc-b189-b37dbac5cbf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:27:55.881171 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27349fcc-2aef-4ecc-b189-b37dbac5cbf9 None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:55.881565 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27349fcc-2aef-4ecc-b189-b37dbac5cbf9 None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:55.881798 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27349fcc-2aef-4ecc-b189-b37dbac5cbf9 None admin] PATH_INFO: `/v3/domains` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:55.882673 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27349fcc-2aef-4ecc-b189-b37dbac5cbf9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80312) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:27:55.883074 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27349fcc-2aef-4ecc-b189-b37dbac5cbf9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:27:55.884613 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27349fcc-2aef-4ecc-b189-b37dbac5cbf9 None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:27:55.893585 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8baf8efd-113b-4eae-aefa-68cc220e5b02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:27:55.894256 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8baf8efd-113b-4eae-aefa-68cc220e5b02 None admin] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:55.894575 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8baf8efd-113b-4eae-aefa-68cc220e5b02 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:55.894855 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8baf8efd-113b-4eae-aefa-68cc220e5b02 None admin] PATH_INFO: `/v3/services` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:55.895586 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8baf8efd-113b-4eae-aefa-68cc220e5b02 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:27:55.896997 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8baf8efd-113b-4eae-aefa-68cc220e5b02 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:27:55.901711 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 382/744] 10.222.0.57 () {56 vars in 1109 bytes} [Fri Jul 1 16:27:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:27:55.901711 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27349fcc-2aef-4ecc-b189-b37dbac5cbf9 None admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:27:55.922374 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 363/745] 10.222.0.57 () {60 vars in 1141 bytes} [Fri Jul 1 16:27:55 2022] POST /identity/v3/services => generated 240 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 16:27:55.922374 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8baf8efd-113b-4eae-aefa-68cc220e5b02 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:27:56.059123 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:27:56.065634 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27349fcc-2aef-4ecc-b189-b37dbac5cbf9 None admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:27:56.075776 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8baf8efd-113b-4eae-aefa-68cc220e5b02 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:27:56.112986 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b44ee3-5716-44e3-91c8-b25f51d51cc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:27:56.113982 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b44ee3-5716-44e3-91c8-b25f51d51cc9 None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:56.114267 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b44ee3-5716-44e3-91c8-b25f51d51cc9 None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:56.114673 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b44ee3-5716-44e3-91c8-b25f51d51cc9 None admin] PATH_INFO: `/v3/domains` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:56.116153 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b44ee3-5716-44e3-91c8-b25f51d51cc9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80312) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:27:56.116470 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b44ee3-5716-44e3-91c8-b25f51d51cc9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:27:56.117858 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b44ee3-5716-44e3-91c8-b25f51d51cc9 None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:27:56.125176 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbd70e9-d544-43cc-ab15-74e1f3e81bc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:27:56.126334 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbd70e9-d544-43cc-ab15-74e1f3e81bc0 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:56.126641 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbd70e9-d544-43cc-ab15-74e1f3e81bc0 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:56.126918 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbd70e9-d544-43cc-ab15-74e1f3e81bc0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:56.134923 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 383/746] 10.222.0.57 () {56 vars in 1109 bytes} [Fri Jul 1 16:27:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:27:56.136075 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b44ee3-5716-44e3-91c8-b25f51d51cc9 None admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:27:56.147551 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bbd70e9-d544-43cc-ab15-74e1f3e81bc0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:27:56.149036 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bbd70e9-d544-43cc-ab15-74e1f3e81bc0 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:27:56.165450 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=100 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:27:56.169794 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application [None req-8bbd70e9-d544-43cc-ab15-74e1f3e81bc0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 16:27:56.169794 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 16:27:56.169794 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 16:27:56.169794 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 16:27:56.169794 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 16:27:56.169794 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 16:27:56.169794 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 16:27:56.169794 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 16:27:56.169794 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 16:27:56.169794 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 16:27:56.169794 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 16:27:56.169794 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 16:27:56.169794 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 01 16:27:56.169794 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 01 16:27:56.169794 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 01 16:27:56.170751 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 01 16:27:56.170751 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 16:27:56.170751 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 16:27:56.170751 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 16:27:56.170751 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 16:27:56.170751 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 16:27:56.170751 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 16:27:56.170751 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 16:27:56.170751 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application with Lock( Jul 01 16:27:56.170751 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 16:27:56.170751 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._enter() Jul 01 16:27:56.170751 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 16:27:56.170751 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 16:27:56.170751 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 16:27:56.170751 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.creator() Jul 01 16:27:56.170751 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 16:27:56.170751 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application created_value = creator( Jul 01 16:27:56.171812 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 01 16:27:56.171812 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 01 16:27:56.171812 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 16:27:56.171812 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Jul 01 16:27:56.176392 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 364/747] 10.222.0.57 () {56 vars in 1108 bytes} [Fri Jul 1 16:27:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 16:27:56.177353 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbd70e9-d544-43cc-ab15-74e1f3e81bc0 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:27:56.268737 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b44ee3-5716-44e3-91c8-b25f51d51cc9 None admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:27:56.297211 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbd70e9-d544-43cc-ab15-74e1f3e81bc0 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:27:56.316649 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0895913d-10f0-466d-bb76-fa4defbd5de5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:27:56.317367 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0895913d-10f0-466d-bb76-fa4defbd5de5 None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:56.317649 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0895913d-10f0-466d-bb76-fa4defbd5de5 None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:56.317926 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0895913d-10f0-466d-bb76-fa4defbd5de5 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:56.329270 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0895913d-10f0-466d-bb76-fa4defbd5de5 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:27:56.331272 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0895913d-10f0-466d-bb76-fa4defbd5de5 None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:27:56.343319 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2b398e-83fb-4f11-9fd8-72470d063b60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:27:56.344775 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2b398e-83fb-4f11-9fd8-72470d063b60 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:56.345262 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2b398e-83fb-4f11-9fd8-72470d063b60 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:56.345732 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2b398e-83fb-4f11-9fd8-72470d063b60 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:56.352751 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application [None req-0895913d-10f0-466d-bb76-fa4defbd5de5 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 01 16:27:56.352751 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 16:27:56.352751 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 16:27:56.352751 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 16:27:56.352751 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 16:27:56.352751 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 16:27:56.352751 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 16:27:56.352751 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 16:27:56.352751 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 16:27:56.352751 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 16:27:56.352751 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 16:27:56.352751 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 16:27:56.352751 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 01 16:27:56.352751 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 01 16:27:56.352751 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 01 16:27:56.352751 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 01 16:27:56.353765 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 16:27:56.353765 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 16:27:56.353765 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 01 16:27:56.353765 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 01 16:27:56.353765 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 01 16:27:56.353765 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 01 16:27:56.353765 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 16:27:56.353765 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 16:27:56.353765 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 16:27:56.353765 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 16:27:56.353765 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 16:27:56.353765 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application with Lock( Jul 01 16:27:56.353765 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 16:27:56.353765 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self._enter() Jul 01 16:27:56.353765 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 16:27:56.353765 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 16:27:56.353765 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 16:27:56.353765 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.creator() Jul 01 16:27:56.354757 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 16:27:56.354757 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application created_value = creator( Jul 01 16:27:56.354757 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 01 16:27:56.354757 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 01 16:27:56.354757 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 01 16:27:56.354757 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 01 16:27:56.354757 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 01 16:27:56.354757 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 01 16:27:56.354757 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 01 16:27:56.354757 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application Jul 01 16:27:56.358107 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f2b398e-83fb-4f11-9fd8-72470d063b60 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:27:56.358638 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 384/748] 10.222.0.57 () {56 vars in 1093 bytes} [Fri Jul 1 16:27:56 2022] GET /identity/v3/users/nova => generated 82 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 16:27:56.359524 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0895913d-10f0-466d-bb76-fa4defbd5de5 None admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:27:56.360308 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f2b398e-83fb-4f11-9fd8-72470d063b60 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:27:56.379851 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application [None req-1f2b398e-83fb-4f11-9fd8-72470d063b60 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 01 16:27:56.379851 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 16:27:56.379851 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 16:27:56.379851 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 16:27:56.379851 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 16:27:56.379851 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 16:27:56.379851 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 16:27:56.379851 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 16:27:56.379851 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 16:27:56.379851 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 16:27:56.379851 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 16:27:56.379851 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 16:27:56.379851 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 01 16:27:56.379851 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 01 16:27:56.379851 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 01 16:27:56.379851 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 01 16:27:56.380924 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 16:27:56.380924 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 16:27:56.380924 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 01 16:27:56.380924 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 01 16:27:56.380924 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 01 16:27:56.380924 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 01 16:27:56.380924 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 16:27:56.380924 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 16:27:56.380924 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 16:27:56.380924 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 16:27:56.380924 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 16:27:56.380924 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application with Lock( Jul 01 16:27:56.380924 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 16:27:56.380924 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._enter() Jul 01 16:27:56.380924 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 16:27:56.380924 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 16:27:56.380924 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 16:27:56.380924 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.creator() Jul 01 16:27:56.381870 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 16:27:56.381870 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application created_value = creator( Jul 01 16:27:56.381870 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 01 16:27:56.381870 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 01 16:27:56.381870 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 01 16:27:56.381870 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 01 16:27:56.381870 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 01 16:27:56.381870 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 01 16:27:56.381870 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 01 16:27:56.381870 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Jul 01 16:27:56.383857 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 365/749] 10.222.0.57 () {56 vars in 1096 bytes} [Fri Jul 1 16:27:56 2022] GET /identity/v3/users/swift => generated 83 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 16:27:56.388959 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2b398e-83fb-4f11-9fd8-72470d063b60 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:27:56.402718 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:27:56.477119 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=100 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:27:56.503038 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2b398e-83fb-4f11-9fd8-72470d063b60 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:27:56.506372 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0895913d-10f0-466d-bb76-fa4defbd5de5 None admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:27:56.544122 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b751cb9a-830d-4970-b0fc-155b383c226c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:27:56.545613 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b751cb9a-830d-4970-b0fc-155b383c226c None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:56.546313 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b751cb9a-830d-4970-b0fc-155b383c226c None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:56.546819 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b751cb9a-830d-4970-b0fc-155b383c226c None admin] PATH_INFO: `/v3/users` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:56.548478 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b751cb9a-830d-4970-b0fc-155b383c226c None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:27:56.549110 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b751cb9a-830d-4970-b0fc-155b383c226c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:27:56.551060 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b751cb9a-830d-4970-b0fc-155b383c226c None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:27:56.553603 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa771c65-fde8-4050-85b3-351a7b2817e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:27:56.554576 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa771c65-fde8-4050-85b3-351a7b2817e9 None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:56.554975 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa771c65-fde8-4050-85b3-351a7b2817e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:56.555326 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa771c65-fde8-4050-85b3-351a7b2817e9 None admin] PATH_INFO: `/v3/domains` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:56.556567 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa771c65-fde8-4050-85b3-351a7b2817e9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80312) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:27:56.556567 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa771c65-fde8-4050-85b3-351a7b2817e9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:27:56.557973 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa771c65-fde8-4050-85b3-351a7b2817e9 None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:27:56.571651 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 385/750] 10.222.0.57 () {56 vars in 1109 bytes} [Fri Jul 1 16:27:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:27:56.572604 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa771c65-fde8-4050-85b3-351a7b2817e9 None admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:27:56.575591 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 366/751] 10.222.0.57 () {56 vars in 1133 bytes} [Fri Jul 1 16:27:56 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:27:56.576596 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b751cb9a-830d-4970-b0fc-155b383c226c None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:27:56.665577 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:27:56.689759 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa771c65-fde8-4050-85b3-351a7b2817e9 None admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:27:56.707101 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b751cb9a-830d-4970-b0fc-155b383c226c None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:27:56.728743 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb683c58-fdc8-4966-b6d6-0fe9065a5eba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:27:56.729561 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb683c58-fdc8-4966-b6d6-0fe9065a5eba None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:56.729943 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb683c58-fdc8-4966-b6d6-0fe9065a5eba None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:56.730333 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb683c58-fdc8-4966-b6d6-0fe9065a5eba None admin] PATH_INFO: `/v3/users` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:56.731735 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb683c58-fdc8-4966-b6d6-0fe9065a5eba None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80312) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:27:56.732147 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb683c58-fdc8-4966-b6d6-0fe9065a5eba None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:27:56.734145 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb683c58-fdc8-4966-b6d6-0fe9065a5eba None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:27:56.746204 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=100 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:27:56.754190 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b2cca1-0013-4b10-bcc5-6591f5d08ae2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:27:56.755683 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b2cca1-0013-4b10-bcc5-6591f5d08ae2 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:56.756799 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b2cca1-0013-4b10-bcc5-6591f5d08ae2 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:56.757420 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b2cca1-0013-4b10-bcc5-6591f5d08ae2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:56.762793 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 386/752] 10.222.0.57 () {56 vars in 1135 bytes} [Fri Jul 1 16:27:56 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:27:56.763880 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb683c58-fdc8-4966-b6d6-0fe9065a5eba None admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:27:56.766048 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b2cca1-0013-4b10-bcc5-6591f5d08ae2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:27:56.769084 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b2cca1-0013-4b10-bcc5-6591f5d08ae2 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:27:56.788727 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application [None req-f8b2cca1-0013-4b10-bcc5-6591f5d08ae2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 01 16:27:56.788727 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 16:27:56.788727 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 16:27:56.788727 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 16:27:56.788727 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 16:27:56.788727 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 16:27:56.788727 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 16:27:56.788727 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 16:27:56.788727 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 16:27:56.788727 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 16:27:56.788727 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 16:27:56.788727 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 16:27:56.788727 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 01 16:27:56.788727 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 01 16:27:56.788727 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 01 16:27:56.789826 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 01 16:27:56.789826 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 16:27:56.789826 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 16:27:56.789826 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 16:27:56.789826 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 16:27:56.789826 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 16:27:56.789826 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 16:27:56.789826 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 16:27:56.789826 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application with Lock( Jul 01 16:27:56.789826 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 16:27:56.789826 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._enter() Jul 01 16:27:56.789826 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 16:27:56.789826 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 16:27:56.789826 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 16:27:56.789826 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.creator() Jul 01 16:27:56.789826 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 16:27:56.789826 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application created_value = creator( Jul 01 16:27:56.790754 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 01 16:27:56.790754 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 01 16:27:56.790754 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 01 16:27:56.790754 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 01 16:27:56.790754 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 01 16:27:56.790754 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 01 16:27:56.790754 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 01 16:27:56.790754 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Jul 01 16:27:56.791295 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 367/753] 10.222.0.57 () {56 vars in 1111 bytes} [Fri Jul 1 16:27:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 16:27:56.792622 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b2cca1-0013-4b10-bcc5-6591f5d08ae2 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:27:56.900050 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb683c58-fdc8-4966-b6d6-0fe9065a5eba None admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:27:56.920014 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b2cca1-0013-4b10-bcc5-6591f5d08ae2 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:27:56.947034 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5603cfa-a90c-4414-a708-6389daf487e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:27:56.947990 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5603cfa-a90c-4414-a708-6389daf487e8 None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:56.948487 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5603cfa-a90c-4414-a708-6389daf487e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:56.948830 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5603cfa-a90c-4414-a708-6389daf487e8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:56.949547 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:27:56.960162 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17a427a-41c7-4267-9456-9e8cbfb64cba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:27:56.961216 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17a427a-41c7-4267-9456-9e8cbfb64cba None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:56.961739 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17a427a-41c7-4267-9456-9e8cbfb64cba None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:56.962240 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17a427a-41c7-4267-9456-9e8cbfb64cba None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:56.962557 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5603cfa-a90c-4414-a708-6389daf487e8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:27:56.964016 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5603cfa-a90c-4414-a708-6389daf487e8 None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:27:56.987325 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application [None req-f5603cfa-a90c-4414-a708-6389daf487e8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 16:27:56.987325 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 16:27:56.987325 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 16:27:56.987325 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 16:27:56.987325 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 16:27:56.987325 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 16:27:56.987325 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 16:27:56.987325 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 16:27:56.987325 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 16:27:56.987325 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 16:27:56.987325 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 16:27:56.987325 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 16:27:56.987325 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 01 16:27:56.987325 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 01 16:27:56.987325 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 01 16:27:56.988309 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 01 16:27:56.988309 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 16:27:56.988309 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 16:27:56.988309 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 16:27:56.988309 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 16:27:56.988309 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 16:27:56.988309 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 16:27:56.988309 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 16:27:56.988309 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application with Lock( Jul 01 16:27:56.988309 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 16:27:56.988309 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self._enter() Jul 01 16:27:56.988309 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 16:27:56.988309 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 16:27:56.988309 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 16:27:56.988309 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.creator() Jul 01 16:27:56.988309 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 16:27:56.988309 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application created_value = creator( Jul 01 16:27:56.989225 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 01 16:27:56.989225 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 01 16:27:56.989225 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 16:27:56.989225 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application Jul 01 16:27:56.993922 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 387/754] 10.222.0.57 () {56 vars in 1108 bytes} [Fri Jul 1 16:27:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 16:27:56.994988 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5603cfa-a90c-4414-a708-6389daf487e8 None admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:27:57.045135 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f17a427a-41c7-4267-9456-9e8cbfb64cba None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:27:57.047069 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f17a427a-41c7-4267-9456-9e8cbfb64cba None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:27:57.063293 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=100 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:27:57.126556 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5603cfa-a90c-4414-a708-6389daf487e8 None admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:27:57.190812 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 368/755] 10.222.0.57 () {58 vars in 1282 bytes} [Fri Jul 1 16:27:56 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 398 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 01 16:27:57.192222 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17a427a-41c7-4267-9456-9e8cbfb64cba None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:27:57.232112 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:27:57.301275 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb6ac15-e7ce-4925-8dc0-2500fdc90670 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:27:57.302369 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb6ac15-e7ce-4925-8dc0-2500fdc90670 None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:57.303093 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb6ac15-e7ce-4925-8dc0-2500fdc90670 None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:57.303694 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb6ac15-e7ce-4925-8dc0-2500fdc90670 None admin] PATH_INFO: `/v3/projects` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:57.304791 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb6ac15-e7ce-4925-8dc0-2500fdc90670 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80312) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:27:57.305323 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb6ac15-e7ce-4925-8dc0-2500fdc90670 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:27:57.307179 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb6ac15-e7ce-4925-8dc0-2500fdc90670 None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:27:57.319040 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17a427a-41c7-4267-9456-9e8cbfb64cba None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:27:57.322452 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 388/756] 10.222.0.57 () {56 vars in 1148 bytes} [Fri Jul 1 16:27:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 328 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:27:57.323619 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb6ac15-e7ce-4925-8dc0-2500fdc90670 None admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:27:57.362564 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024ae391-f018-4b43-a196-4bb3bc204d7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:27:57.363246 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024ae391-f018-4b43-a196-4bb3bc204d7f None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:57.363535 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024ae391-f018-4b43-a196-4bb3bc204d7f None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:57.363817 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024ae391-f018-4b43-a196-4bb3bc204d7f None admin] PATH_INFO: `/v3/domains` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:57.366553 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024ae391-f018-4b43-a196-4bb3bc204d7f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:27:57.366880 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024ae391-f018-4b43-a196-4bb3bc204d7f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:27:57.368267 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024ae391-f018-4b43-a196-4bb3bc204d7f None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:27:57.386502 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 369/757] 10.222.0.57 () {56 vars in 1109 bytes} [Fri Jul 1 16:27:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:27:57.387656 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024ae391-f018-4b43-a196-4bb3bc204d7f None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:27:57.449930 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb6ac15-e7ce-4925-8dc0-2500fdc90670 None admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:27:57.464817 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=100 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:27:57.492284 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:27:57.494021 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9c0933-fe5e-45f3-a1fb-d1ede4df836a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:27:57.495167 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9c0933-fe5e-45f3-a1fb-d1ede4df836a None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:57.495462 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9c0933-fe5e-45f3-a1fb-d1ede4df836a None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:57.495743 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9c0933-fe5e-45f3-a1fb-d1ede4df836a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:57.508819 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024ae391-f018-4b43-a196-4bb3bc204d7f None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:27:57.509831 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a9c0933-fe5e-45f3-a1fb-d1ede4df836a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:27:57.511687 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a9c0933-fe5e-45f3-a1fb-d1ede4df836a None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:27:57.531578 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application [None req-7a9c0933-fe5e-45f3-a1fb-d1ede4df836a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 16:27:57.531578 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 16:27:57.531578 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 16:27:57.531578 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 16:27:57.531578 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 16:27:57.531578 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 16:27:57.531578 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 16:27:57.531578 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 16:27:57.531578 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 16:27:57.531578 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 16:27:57.531578 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 16:27:57.531578 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 16:27:57.531578 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 01 16:27:57.531578 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 01 16:27:57.531578 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 01 16:27:57.532545 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 01 16:27:57.532545 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 16:27:57.532545 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 16:27:57.532545 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 16:27:57.532545 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 16:27:57.532545 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 16:27:57.532545 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 16:27:57.532545 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 16:27:57.532545 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application with Lock( Jul 01 16:27:57.532545 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 16:27:57.532545 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self._enter() Jul 01 16:27:57.532545 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 16:27:57.532545 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 16:27:57.532545 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 16:27:57.532545 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.creator() Jul 01 16:27:57.532545 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 16:27:57.532545 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application created_value = creator( Jul 01 16:27:57.533423 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 01 16:27:57.533423 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 01 16:27:57.533423 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 16:27:57.533423 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application Jul 01 16:27:57.538126 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 389/758] 10.222.0.57 () {56 vars in 1108 bytes} [Fri Jul 1 16:27:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 16:27:57.539143 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9c0933-fe5e-45f3-a1fb-d1ede4df836a None admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:27:57.550755 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470880d4-4586-44ef-964d-895324d128b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:27:57.551703 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470880d4-4586-44ef-964d-895324d128b4 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:57.552270 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470880d4-4586-44ef-964d-895324d128b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:57.553362 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470880d4-4586-44ef-964d-895324d128b4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:57.554583 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-470880d4-4586-44ef-964d-895324d128b4 None admin] RBAC: Adding query filter params (role.id=e8809b67c9dc471daa1c2bd42c9a2baf, scope.project.id=e0788e6725a84e57a4174284c1845755, user.id=c4de1b1eeaed4fc5ae7200b2c7a90957) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:27:57.555205 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-470880d4-4586-44ef-964d-895324d128b4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:27:57.557014 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-470880d4-4586-44ef-964d-895324d128b4 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:27:57.563967 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 370/759] 10.222.0.57 () {56 vars in 1374 bytes} [Fri Jul 1 16:27:57 2022] GET /identity/v3/role_assignments?user.id=c4de1b1eeaed4fc5ae7200b2c7a90957&scope.project.id=e0788e6725a84e57a4174284c1845755&role.id=e8809b67c9dc471daa1c2bd42c9a2baf => generated 259 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:27:57.565038 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470880d4-4586-44ef-964d-895324d128b4 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:27:57.658549 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9c0933-fe5e-45f3-a1fb-d1ede4df836a None admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:27:57.697795 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5edbcd83-07ea-46fe-b2f0-6899d7e7423a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:27:57.698664 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edbcd83-07ea-46fe-b2f0-6899d7e7423a None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:57.698953 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edbcd83-07ea-46fe-b2f0-6899d7e7423a None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:57.699235 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edbcd83-07ea-46fe-b2f0-6899d7e7423a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:57.707053 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5edbcd83-07ea-46fe-b2f0-6899d7e7423a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:27:57.708669 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5edbcd83-07ea-46fe-b2f0-6899d7e7423a None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:27:57.712984 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470880d4-4586-44ef-964d-895324d128b4 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:27:57.724417 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application [None req-5edbcd83-07ea-46fe-b2f0-6899d7e7423a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 01 16:27:57.724417 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 16:27:57.724417 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 16:27:57.724417 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 16:27:57.724417 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 16:27:57.724417 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 16:27:57.724417 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 16:27:57.724417 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 16:27:57.724417 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 16:27:57.724417 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 16:27:57.724417 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 16:27:57.724417 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 16:27:57.724417 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 01 16:27:57.724417 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 01 16:27:57.724417 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 01 16:27:57.725731 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 01 16:27:57.725731 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 16:27:57.725731 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 16:27:57.725731 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 16:27:57.725731 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 16:27:57.725731 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 16:27:57.725731 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 16:27:57.725731 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 16:27:57.725731 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application with Lock( Jul 01 16:27:57.725731 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 16:27:57.725731 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self._enter() Jul 01 16:27:57.725731 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 16:27:57.725731 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 16:27:57.725731 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 16:27:57.725731 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.creator() Jul 01 16:27:57.725731 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 16:27:57.725731 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application created_value = creator( Jul 01 16:27:57.726651 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 01 16:27:57.726651 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 01 16:27:57.726651 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 01 16:27:57.726651 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 01 16:27:57.726651 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 01 16:27:57.726651 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 01 16:27:57.726651 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 01 16:27:57.726651 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application Jul 01 16:27:57.730653 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 390/760] 10.222.0.57 () {56 vars in 1111 bytes} [Fri Jul 1 16:27:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 16:27:57.731969 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5edbcd83-07ea-46fe-b2f0-6899d7e7423a None admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:27:57.759177 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b572d8f-5a50-428f-a8d3-031acc784510 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:27:57.760267 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b572d8f-5a50-428f-a8d3-031acc784510 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:57.760762 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=100 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:27:57.761268 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b572d8f-5a50-428f-a8d3-031acc784510 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:57.762478 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b572d8f-5a50-428f-a8d3-031acc784510 None admin] PATH_INFO: `/v3/domains` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:57.763573 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b572d8f-5a50-428f-a8d3-031acc784510 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:27:57.764124 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b572d8f-5a50-428f-a8d3-031acc784510 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:27:57.766635 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b572d8f-5a50-428f-a8d3-031acc784510 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:27:57.783177 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 371/761] 10.222.0.57 () {56 vars in 1109 bytes} [Fri Jul 1 16:27:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:27:57.807939 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b572d8f-5a50-428f-a8d3-031acc784510 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:27:57.847153 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:27:57.862233 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5edbcd83-07ea-46fe-b2f0-6899d7e7423a None admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:27:57.900093 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b541a30-8008-4f16-affc-c3539f86b7d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:27:57.900764 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b541a30-8008-4f16-affc-c3539f86b7d3 None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:57.901051 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b541a30-8008-4f16-affc-c3539f86b7d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:57.901331 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b541a30-8008-4f16-affc-c3539f86b7d3 None admin] PATH_INFO: `/v3/projects` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:57.902133 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b541a30-8008-4f16-affc-c3539f86b7d3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80312) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:27:57.902480 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b541a30-8008-4f16-affc-c3539f86b7d3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:27:57.903938 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b541a30-8008-4f16-affc-c3539f86b7d3 None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:27:57.922097 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 391/762] 10.222.0.57 () {56 vars in 1148 bytes} [Fri Jul 1 16:27:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 01 16:27:57.923318 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b541a30-8008-4f16-affc-c3539f86b7d3 None admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:27:58.032481 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=100 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:27:58.069046 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b541a30-8008-4f16-affc-c3539f86b7d3 None admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:27:58.130116 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b0277a-87e9-43bc-a2b3-3cee0c766b57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:27:58.132216 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b0277a-87e9-43bc-a2b3-3cee0c766b57 None admin] REQUEST_METHOD: `PUT` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:58.132711 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b0277a-87e9-43bc-a2b3-3cee0c766b57 None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:58.133204 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b0277a-87e9-43bc-a2b3-3cee0c766b57 None admin] PATH_INFO: `/v3/projects/e0788e6725a84e57a4174284c1845755/users/7a50f745b31941b7a7265c0fc852f48e/roles/84ee739146934c1ab32c501ff8093644` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:58.136700 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b0277a-87e9-43bc-a2b3-3cee0c766b57 None admin] RBAC: Authorizing `identity:create_grant(project_id=e0788e6725a84e57a4174284c1845755, user_id=7a50f745b31941b7a7265c0fc852f48e, role_id=84ee739146934c1ab32c501ff8093644)` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:27:58.138096 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b572d8f-5a50-428f-a8d3-031acc784510 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:27:58.139251 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b0277a-87e9-43bc-a2b3-3cee0c766b57 None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:27:58.170570 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 392/763] 10.222.0.57 () {58 vars in 1439 bytes} [Fri Jul 1 16:27:57 2022] PUT /identity/v3/projects/e0788e6725a84e57a4174284c1845755/users/7a50f745b31941b7a7265c0fc852f48e/roles/84ee739146934c1ab32c501ff8093644 => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 16:27:58.173468 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae16cf6-db78-4a77-bf03-e14fdbdf98de None None] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:58.173948 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae16cf6-db78-4a77-bf03-e14fdbdf98de None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:58.174492 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae16cf6-db78-4a77-bf03-e14fdbdf98de None None] PATH_INFO: `/` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:58.175482 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f90f48-f27e-4d71-b5b9-b775ee10233f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:27:58.176395 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f90f48-f27e-4d71-b5b9-b775ee10233f None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:58.176916 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f90f48-f27e-4d71-b5b9-b775ee10233f None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:58.177336 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f90f48-f27e-4d71-b5b9-b775ee10233f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:58.181636 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 393/764] 10.222.0.57 () {54 vars in 926 bytes} [Fri Jul 1 16:27:58 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 16:27:58.184396 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8ec059-623f-4db2-82f1-c0c292b7d986 None None] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:58.184897 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8ec059-623f-4db2-82f1-c0c292b7d986 None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:58.185397 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8ec059-623f-4db2-82f1-c0c292b7d986 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:58.185558 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f90f48-f27e-4d71-b5b9-b775ee10233f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:27:58.187142 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f90f48-f27e-4d71-b5b9-b775ee10233f None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:27:58.205341 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application [None req-80f90f48-f27e-4d71-b5b9-b775ee10233f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 01 16:27:58.205341 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 16:27:58.205341 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 16:27:58.205341 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 16:27:58.205341 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 16:27:58.205341 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 16:27:58.205341 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 16:27:58.205341 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 16:27:58.205341 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 16:27:58.205341 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 16:27:58.205341 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 16:27:58.205341 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 16:27:58.205341 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 01 16:27:58.205341 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 01 16:27:58.205341 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 01 16:27:58.208966 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 01 16:27:58.208966 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 16:27:58.208966 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 16:27:58.208966 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 16:27:58.208966 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 16:27:58.208966 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 16:27:58.208966 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 16:27:58.208966 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 16:27:58.208966 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application with Lock( Jul 01 16:27:58.208966 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 16:27:58.208966 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._enter() Jul 01 16:27:58.208966 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 16:27:58.208966 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 16:27:58.208966 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 16:27:58.208966 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.creator() Jul 01 16:27:58.208966 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 16:27:58.208966 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application created_value = creator( Jul 01 16:27:58.212104 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 01 16:27:58.212104 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 01 16:27:58.212104 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 01 16:27:58.212104 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 01 16:27:58.212104 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 01 16:27:58.212104 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 01 16:27:58.212104 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 01 16:27:58.212104 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Jul 01 16:27:58.212104 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 372/765] 10.222.0.57 () {56 vars in 1111 bytes} [Fri Jul 1 16:27:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 400 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 01 16:27:58.212104 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b020c96-9817-4256-8551-9b6b7106b862 None None] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:58.212104 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b020c96-9817-4256-8551-9b6b7106b862 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:58.213449 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b020c96-9817-4256-8551-9b6b7106b862 None None] PATH_INFO: `/` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:58.213449 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 373/766] 10.222.0.57 () {54 vars in 926 bytes} [Fri Jul 1 16:27:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 16:27:58.222718 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b020c96-9817-4256-8551-9b6b7106b862 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:27:58.237140 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.auth.core [None req-8f8ec059-623f-4db2-82f1-c0c292b7d986 None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80312) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:27:58.237140 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-8f8ec059-623f-4db2-82f1-c0c292b7d986 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:27:58.316977 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 394/767] 10.222.0.57 () {58 vars in 1023 bytes} [Fri Jul 1 16:27:58 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 01 16:27:58.318947 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18556a20-7daa-4601-a118-aaf50ca4d9cc None None] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:58.319433 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18556a20-7daa-4601-a118-aaf50ca4d9cc None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:58.319862 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18556a20-7daa-4601-a118-aaf50ca4d9cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:58.351680 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:27:58.353981 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.auth.core [None req-18556a20-7daa-4601-a118-aaf50ca4d9cc None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80312) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:27:58.355572 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-18556a20-7daa-4601-a118-aaf50ca4d9cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:27:58.406233 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b020c96-9817-4256-8551-9b6b7106b862 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:27:58.437216 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 395/768] 10.222.0.57 () {58 vars in 1023 bytes} [Fri Jul 1 16:27:58 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 01 16:27:58.438007 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18556a20-7daa-4601-a118-aaf50ca4d9cc None None] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:27:58.445397 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:27:58.458067 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6be321a-e42a-4f5e-89d7-e148c0da9351 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:27:58.459723 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6be321a-e42a-4f5e-89d7-e148c0da9351 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:58.460218 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6be321a-e42a-4f5e-89d7-e148c0da9351 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:58.461327 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6be321a-e42a-4f5e-89d7-e148c0da9351 None admin] PATH_INFO: `/v3/projects` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:58.462362 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6be321a-e42a-4f5e-89d7-e148c0da9351 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:27:58.462901 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6be321a-e42a-4f5e-89d7-e148c0da9351 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:27:58.464587 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6be321a-e42a-4f5e-89d7-e148c0da9351 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:27:58.481726 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 374/769] 10.222.0.57 () {56 vars in 1148 bytes} [Fri Jul 1 16:27:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:27:58.492292 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6be321a-e42a-4f5e-89d7-e148c0da9351 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:27:58.581477 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18556a20-7daa-4601-a118-aaf50ca4d9cc None None] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:27:58.626907 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5fa368-104d-4241-8943-503ffabe2d5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:27:58.627767 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5fa368-104d-4241-8943-503ffabe2d5e None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:58.628007 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5fa368-104d-4241-8943-503ffabe2d5e None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:58.628288 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5fa368-104d-4241-8943-503ffabe2d5e None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:58.628948 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a5fa368-104d-4241-8943-503ffabe2d5e None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:27:58.630318 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a5fa368-104d-4241-8943-503ffabe2d5e None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:27:58.641058 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6be321a-e42a-4f5e-89d7-e148c0da9351 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:27:58.642979 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application [None req-6a5fa368-104d-4241-8943-503ffabe2d5e None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 01 16:27:58.642979 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 16:27:58.642979 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 01 16:27:58.642979 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 01 16:27:58.642979 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 01 16:27:58.642979 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 01 16:27:58.642979 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 01 16:27:58.642979 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 01 16:27:58.642979 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 01 16:27:58.642979 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application Jul 01 16:27:58.642979 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 01 16:27:58.642979 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application Jul 01 16:27:58.642979 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 16:27:58.642979 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 16:27:58.642979 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 16:27:58.642979 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 16:27:58.643911 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 16:27:58.643911 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 16:27:58.643911 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 16:27:58.643911 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 16:27:58.643911 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 16:27:58.643911 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 16:27:58.643911 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 16:27:58.643911 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 01 16:27:58.643911 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 01 16:27:58.643911 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 01 16:27:58.643911 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 01 16:27:58.643911 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 16:27:58.643911 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 16:27:58.643911 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 16:27:58.643911 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 16:27:58.643911 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 16:27:58.644792 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 16:27:58.644792 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 16:27:58.644792 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application with Lock( Jul 01 16:27:58.644792 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 16:27:58.644792 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self._enter() Jul 01 16:27:58.644792 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 16:27:58.644792 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 16:27:58.644792 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 16:27:58.644792 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.creator() Jul 01 16:27:58.644792 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 16:27:58.644792 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application created_value = creator( Jul 01 16:27:58.644792 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 01 16:27:58.644792 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 01 16:27:58.644792 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 01 16:27:58.644792 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application Jul 01 16:27:58.650499 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 396/770] 10.222.0.57 () {56 vars in 1129 bytes} [Fri Jul 1 16:27:58 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 16:27:58.651420 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5fa368-104d-4241-8943-503ffabe2d5e None admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:27:58.688610 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:27:58.695841 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99b6935-8115-4002-812c-9f55639d29f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:27:58.697526 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99b6935-8115-4002-812c-9f55639d29f9 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:58.698918 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99b6935-8115-4002-812c-9f55639d29f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:58.699536 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99b6935-8115-4002-812c-9f55639d29f9 None admin] PATH_INFO: `/v3/services/network` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:58.700847 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99b6935-8115-4002-812c-9f55639d29f9 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:27:58.703025 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99b6935-8115-4002-812c-9f55639d29f9 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:27:58.719536 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application [None req-b99b6935-8115-4002-812c-9f55639d29f9 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 01 16:27:58.719536 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 16:27:58.719536 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 01 16:27:58.719536 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 01 16:27:58.719536 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 01 16:27:58.719536 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 01 16:27:58.719536 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 01 16:27:58.719536 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 01 16:27:58.719536 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 01 16:27:58.719536 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Jul 01 16:27:58.719536 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 01 16:27:58.719536 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Jul 01 16:27:58.719536 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 16:27:58.719536 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 16:27:58.719536 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 16:27:58.719536 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 16:27:58.719536 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 16:27:58.721027 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 16:27:58.721027 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 16:27:58.721027 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 16:27:58.721027 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 16:27:58.721027 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 16:27:58.721027 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 16:27:58.721027 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 01 16:27:58.721027 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 01 16:27:58.721027 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 01 16:27:58.721027 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 01 16:27:58.721027 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 16:27:58.721027 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 16:27:58.721027 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 16:27:58.721027 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 16:27:58.721027 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 16:27:58.721027 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 16:27:58.721906 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 16:27:58.721906 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application with Lock( Jul 01 16:27:58.721906 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 16:27:58.721906 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._enter() Jul 01 16:27:58.721906 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 16:27:58.721906 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 16:27:58.721906 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 16:27:58.721906 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.creator() Jul 01 16:27:58.721906 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 16:27:58.721906 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application created_value = creator( Jul 01 16:27:58.721906 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 01 16:27:58.721906 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 01 16:27:58.721906 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 01 16:27:58.721906 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Jul 01 16:27:58.729457 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 375/771] 10.222.0.57 () {56 vars in 1111 bytes} [Fri Jul 1 16:27:58 2022] GET /identity/v3/services/network => generated 88 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 01 16:27:58.729457 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99b6935-8115-4002-812c-9f55639d29f9 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:27:58.738889 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=100 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:27:58.780470 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5fa368-104d-4241-8943-503ffabe2d5e None admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:27:58.821311 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a8111b-0e09-4f1f-9220-30467fffcdbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:27:58.821981 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a8111b-0e09-4f1f-9220-30467fffcdbf None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:58.822272 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a8111b-0e09-4f1f-9220-30467fffcdbf None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:58.822593 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a8111b-0e09-4f1f-9220-30467fffcdbf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:58.823432 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a8111b-0e09-4f1f-9220-30467fffcdbf None admin] RBAC: Adding query filter params (role.id=84ee739146934c1ab32c501ff8093644, scope.project.id=e0788e6725a84e57a4174284c1845755, user.id=859fe410c2d2450fb803c6244839f27c) {{(pid=80312) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:27:58.823747 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a8111b-0e09-4f1f-9220-30467fffcdbf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:27:58.825245 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a8111b-0e09-4f1f-9220-30467fffcdbf None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:27:58.838672 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 397/772] 10.222.0.57 () {56 vars in 1374 bytes} [Fri Jul 1 16:27:58 2022] GET /identity/v3/role_assignments?user.id=859fe410c2d2450fb803c6244839f27c&scope.project.id=e0788e6725a84e57a4174284c1845755&role.id=84ee739146934c1ab32c501ff8093644 => generated 259 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:27:58.839711 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a8111b-0e09-4f1f-9220-30467fffcdbf None admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:27:58.873471 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99b6935-8115-4002-812c-9f55639d29f9 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:27:58.917551 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3700e6a-e2ba-4601-8555-d1dc9bb360a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:27:58.918734 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3700e6a-e2ba-4601-8555-d1dc9bb360a5 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:58.919348 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3700e6a-e2ba-4601-8555-d1dc9bb360a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:58.919831 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3700e6a-e2ba-4601-8555-d1dc9bb360a5 None admin] PATH_INFO: `/v3/services` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:58.920883 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3700e6a-e2ba-4601-8555-d1dc9bb360a5 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:27:58.921441 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3700e6a-e2ba-4601-8555-d1dc9bb360a5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:27:58.923157 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3700e6a-e2ba-4601-8555-d1dc9bb360a5 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:27:58.929588 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 376/773] 10.222.0.57 () {56 vars in 1124 bytes} [Fri Jul 1 16:27:58 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:27:58.939613 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3700e6a-e2ba-4601-8555-d1dc9bb360a5 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:27:59.065202 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:27:59.090091 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a8111b-0e09-4f1f-9220-30467fffcdbf None admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:27:59.130249 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3700e6a-e2ba-4601-8555-d1dc9bb360a5 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:27:59.139591 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=100 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:27:59.153782 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f678f60-ed18-4885-a8a4-6343d1ae4388 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:27:59.154494 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f678f60-ed18-4885-a8a4-6343d1ae4388 None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:59.154777 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f678f60-ed18-4885-a8a4-6343d1ae4388 None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:59.155054 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f678f60-ed18-4885-a8a4-6343d1ae4388 None admin] PATH_INFO: `/v3/services` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:59.155769 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f678f60-ed18-4885-a8a4-6343d1ae4388 None admin] RBAC: Adding query filter params (name=network) {{(pid=80312) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:27:59.156077 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f678f60-ed18-4885-a8a4-6343d1ae4388 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:27:59.157444 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f678f60-ed18-4885-a8a4-6343d1ae4388 None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:27:59.170672 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 398/774] 10.222.0.57 () {56 vars in 1112 bytes} [Fri Jul 1 16:27:58 2022] GET /identity/v3/services?name=network => generated 124 bytes in 331 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:27:59.174357 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f678f60-ed18-4885-a8a4-6343d1ae4388 None admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:27:59.219410 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db7f580-7b6a-47c7-a474-9537d6c24f6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:27:59.220008 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db7f580-7b6a-47c7-a474-9537d6c24f6c None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:59.220289 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db7f580-7b6a-47c7-a474-9537d6c24f6c None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:59.220564 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db7f580-7b6a-47c7-a474-9537d6c24f6c None admin] PATH_INFO: `/v3/services` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:59.221283 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db7f580-7b6a-47c7-a474-9537d6c24f6c None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:27:59.221589 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db7f580-7b6a-47c7-a474-9537d6c24f6c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:27:59.222994 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db7f580-7b6a-47c7-a474-9537d6c24f6c None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:27:59.235383 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 377/775] 10.222.0.57 () {56 vars in 1124 bytes} [Fri Jul 1 16:27:58 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:27:59.243233 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db7f580-7b6a-47c7-a474-9537d6c24f6c None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:27:59.372722 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f678f60-ed18-4885-a8a4-6343d1ae4388 None admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:27:59.398410 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db7f580-7b6a-47c7-a474-9537d6c24f6c None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:27:59.425355 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:27:59.428280 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7339a293-b53f-40c8-8572-d17e6f423061 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:27:59.429201 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7339a293-b53f-40c8-8572-d17e6f423061 None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:59.429666 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7339a293-b53f-40c8-8572-d17e6f423061 None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:59.430108 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7339a293-b53f-40c8-8572-d17e6f423061 None admin] PATH_INFO: `/v3/services` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:59.431029 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7339a293-b53f-40c8-8572-d17e6f423061 None admin] RBAC: Adding query filter params (type=network) {{(pid=80312) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:27:59.432768 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7339a293-b53f-40c8-8572-d17e6f423061 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:27:59.435157 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7339a293-b53f-40c8-8572-d17e6f423061 None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:27:59.447844 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 399/776] 10.222.0.57 () {56 vars in 1112 bytes} [Fri Jul 1 16:27:59 2022] GET /identity/v3/services?type=network => generated 350 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:27:59.448739 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c17e4df-7db8-4c7f-ab23-1847a404ac51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:27:59.449683 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c17e4df-7db8-4c7f-ab23-1847a404ac51 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:59.450141 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c17e4df-7db8-4c7f-ab23-1847a404ac51 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:59.450649 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c17e4df-7db8-4c7f-ab23-1847a404ac51 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:59.451642 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c17e4df-7db8-4c7f-ab23-1847a404ac51 None admin] RBAC: Adding query filter params (interface=public, service_id=2014b3b8d9b9496cb975920ee5803bc0, region_id=RegionOne) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:27:59.452843 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7339a293-b53f-40c8-8572-d17e6f423061 None admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:27:59.453648 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c17e4df-7db8-4c7f-ab23-1847a404ac51 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:27:59.455249 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c17e4df-7db8-4c7f-ab23-1847a404ac51 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:27:59.473140 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 378/777] 10.222.0.57 () {56 vars in 1251 bytes} [Fri Jul 1 16:27:59 2022] GET /identity/v3/endpoints?service_id=2014b3b8d9b9496cb975920ee5803bc0&interface=public®ion_id=RegionOne => generated 194 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:27:59.488113 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c17e4df-7db8-4c7f-ab23-1847a404ac51 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:27:59.572424 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=100 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:27:59.650411 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7339a293-b53f-40c8-8572-d17e6f423061 None admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:27:59.674415 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c17e4df-7db8-4c7f-ab23-1847a404ac51 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:27:59.713188 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4c97a8-57a1-4466-adc8-727cb288fdb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:27:59.714199 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4c97a8-57a1-4466-adc8-727cb288fdb8 None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:59.714684 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4c97a8-57a1-4466-adc8-727cb288fdb8 None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:59.716318 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4c97a8-57a1-4466-adc8-727cb288fdb8 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:59.717302 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d4c97a8-57a1-4466-adc8-727cb288fdb8 None admin] RBAC: Adding query filter params (interface=public, service_id=4e591e5b6bf04b4bae45215673eb25f6, region_id=RegionOne) {{(pid=80312) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:27:59.718601 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d4c97a8-57a1-4466-adc8-727cb288fdb8 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:27:59.720160 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d4c97a8-57a1-4466-adc8-727cb288fdb8 None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:27:59.733857 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 400/778] 10.222.0.57 () {56 vars in 1251 bytes} [Fri Jul 1 16:27:59 2022] GET /identity/v3/endpoints?service_id=4e591e5b6bf04b4bae45215673eb25f6&interface=public®ion_id=RegionOne => generated 194 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:27:59.738238 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b6cc9e-b518-4f00-8af4-fd8ad179f15a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:27:59.739737 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4c97a8-57a1-4466-adc8-727cb288fdb8 None admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:27:59.741474 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b6cc9e-b518-4f00-8af4-fd8ad179f15a None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:59.742361 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b6cc9e-b518-4f00-8af4-fd8ad179f15a None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:59.742823 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b6cc9e-b518-4f00-8af4-fd8ad179f15a None admin] PATH_INFO: `/v3/services` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:59.744078 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b6cc9e-b518-4f00-8af4-fd8ad179f15a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:27:59.747531 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b6cc9e-b518-4f00-8af4-fd8ad179f15a None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:27:59.760108 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 379/779] 10.222.0.57 () {56 vars in 1087 bytes} [Fri Jul 1 16:27:59 2022] GET /identity/v3/services => generated 772 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 01 16:27:59.920023 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4c97a8-57a1-4466-adc8-727cb288fdb8 None admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:27:59.966483 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e278e7d-3164-42f6-84bf-85e86a10d4af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:27:59.967170 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e278e7d-3164-42f6-84bf-85e86a10d4af None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:27:59.967454 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e278e7d-3164-42f6-84bf-85e86a10d4af None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:27:59.967727 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e278e7d-3164-42f6-84bf-85e86a10d4af None admin] PATH_INFO: `/v3/services` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:27:59.968425 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e278e7d-3164-42f6-84bf-85e86a10d4af None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:27:59.969832 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e278e7d-3164-42f6-84bf-85e86a10d4af None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:27:59.971140 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=100 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:27:59.976831 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 401/780] 10.222.0.57 () {56 vars in 1087 bytes} [Fri Jul 1 16:27:59 2022] GET /identity/v3/services => generated 772 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:28:00.028651 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a93975-68e3-4000-a0b9-1c12316eb51f None None] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:28:00.029279 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a93975-68e3-4000-a0b9-1c12316eb51f None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:28:00.029762 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a93975-68e3-4000-a0b9-1c12316eb51f None None] PATH_INFO: `/` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:28:00.031897 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 380/781] 10.222.0.57 () {54 vars in 926 bytes} [Fri Jul 1 16:28:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 16:28:00.040947 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c388647d-db53-4224-9405-c37ac58c375f None None] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:28:00.040947 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c388647d-db53-4224-9405-c37ac58c375f None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:28:00.040947 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c388647d-db53-4224-9405-c37ac58c375f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:28:00.070274 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.auth.core [None req-c388647d-db53-4224-9405-c37ac58c375f None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80312) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:28:00.072134 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-c388647d-db53-4224-9405-c37ac58c375f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:28:00.170997 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 402/782] 10.222.0.57 () {58 vars in 1023 bytes} [Fri Jul 1 16:28:00 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 01 16:28:00.195870 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a93975-68e3-4000-a0b9-1c12316eb51f None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:28:00.275395 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:28:00.348142 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a93975-68e3-4000-a0b9-1c12316eb51f None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:28:00.388761 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1aafda-24d7-4739-add1-163b07686676 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:28:00.389578 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1aafda-24d7-4739-add1-163b07686676 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:28:00.389866 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1aafda-24d7-4739-add1-163b07686676 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:28:00.390148 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1aafda-24d7-4739-add1-163b07686676 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:28:00.411403 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1aafda-24d7-4739-add1-163b07686676 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 01 16:28:00.420402 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application [None req-ff1aafda-24d7-4739-add1-163b07686676 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 01 16:28:00.420402 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 16:28:00.420402 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 16:28:00.420402 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 16:28:00.420402 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 16:28:00.420402 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 16:28:00.420402 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 16:28:00.420402 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 16:28:00.420402 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 16:28:00.420402 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 16:28:00.420402 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 16:28:00.420402 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 16:28:00.420402 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 01 16:28:00.420402 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 01 16:28:00.420402 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 01 16:28:00.420402 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 01 16:28:00.421579 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 01 16:28:00.421579 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 01 16:28:00.421579 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 01 16:28:00.421579 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 01 16:28:00.421579 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 16:28:00.421579 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 16:28:00.421579 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 16:28:00.421579 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 16:28:00.421579 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 16:28:00.421579 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 16:28:00.421579 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 16:28:00.421579 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application with Lock( Jul 01 16:28:00.421579 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 16:28:00.421579 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._enter() Jul 01 16:28:00.421579 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 16:28:00.421579 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 16:28:00.422508 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 16:28:00.422508 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.creator() Jul 01 16:28:00.422508 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 16:28:00.422508 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application created_value = creator( Jul 01 16:28:00.422508 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 01 16:28:00.422508 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 01 16:28:00.422508 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 01 16:28:00.422508 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 01 16:28:00.422508 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 01 16:28:00.422508 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 01 16:28:00.422508 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 01 16:28:00.422508 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Jul 01 16:28:00.422508 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 381/783] 10.222.0.57 () {56 vars in 1102 bytes} [Fri Jul 1 16:28:00 2022] GET /identity/v3/roles/service => generated 85 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 16:28:00.428962 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c388647d-db53-4224-9405-c37ac58c375f None None] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:28:00.570205 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c388647d-db53-4224-9405-c37ac58c375f None None] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:28:00.587928 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=100 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:28:00.610722 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8ba08d-2330-4844-b87d-8c753a7d2863 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:28:00.611661 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8ba08d-2330-4844-b87d-8c753a7d2863 None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:28:00.612075 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8ba08d-2330-4844-b87d-8c753a7d2863 None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:28:00.612464 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8ba08d-2330-4844-b87d-8c753a7d2863 None admin] PATH_INFO: `/v3/roles` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:28:00.613508 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc8ba08d-2330-4844-b87d-8c753a7d2863 None admin] RBAC: Adding query filter params (name=service) {{(pid=80312) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:28:00.614002 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc8ba08d-2330-4844-b87d-8c753a7d2863 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:28:00.616423 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc8ba08d-2330-4844-b87d-8c753a7d2863 None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:28:00.628797 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 403/784] 10.222.0.57 () {56 vars in 1103 bytes} [Fri Jul 1 16:28:00 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:28:00.634512 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1aafda-24d7-4739-add1-163b07686676 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:28:00.689739 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe638ea7-5568-4213-880e-25b0f3d87abc None None] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:28:00.690067 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe638ea7-5568-4213-880e-25b0f3d87abc None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:28:00.690294 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe638ea7-5568-4213-880e-25b0f3d87abc None None] PATH_INFO: `/` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:28:00.692167 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 404/785] 10.222.0.57 () {54 vars in 926 bytes} [Fri Jul 1 16:28:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 16:28:00.698158 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f72daa-7b56-4baa-bfab-18d4235a1e68 None None] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:28:00.698473 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f72daa-7b56-4baa-bfab-18d4235a1e68 None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:28:00.698755 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f72daa-7b56-4baa-bfab-18d4235a1e68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:28:00.716944 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.auth.core [None req-36f72daa-7b56-4baa-bfab-18d4235a1e68 None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80312) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:28:00.718139 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-36f72daa-7b56-4baa-bfab-18d4235a1e68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:28:00.824929 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 405/786] 10.222.0.57 () {58 vars in 1023 bytes} [Fri Jul 1 16:28:00 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 01 16:28:00.837194 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1aafda-24d7-4739-add1-163b07686676 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:28:00.867343 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:28:00.895051 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f72daa-7b56-4baa-bfab-18d4235a1e68 None None] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:28:00.897778 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdbf8615-0649-4898-a264-673c046223c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:28:00.898799 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbf8615-0649-4898-a264-673c046223c8 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:28:00.899078 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbf8615-0649-4898-a264-673c046223c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:28:00.899397 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbf8615-0649-4898-a264-673c046223c8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:28:00.910839 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdbf8615-0649-4898-a264-673c046223c8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:28:00.912768 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdbf8615-0649-4898-a264-673c046223c8 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:28:00.933061 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application [None req-fdbf8615-0649-4898-a264-673c046223c8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 16:28:00.933061 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 16:28:00.933061 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 16:28:00.933061 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 16:28:00.933061 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 16:28:00.933061 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 16:28:00.933061 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 16:28:00.933061 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 16:28:00.933061 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 16:28:00.933061 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 16:28:00.933061 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 16:28:00.933061 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 16:28:00.933061 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 01 16:28:00.933061 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 01 16:28:00.933061 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 01 16:28:00.934015 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 01 16:28:00.934015 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 16:28:00.934015 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 16:28:00.934015 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 16:28:00.934015 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 16:28:00.934015 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 16:28:00.934015 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 16:28:00.934015 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 16:28:00.934015 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application with Lock( Jul 01 16:28:00.934015 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 16:28:00.934015 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._enter() Jul 01 16:28:00.934015 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 16:28:00.934015 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 16:28:00.934015 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 16:28:00.934015 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.creator() Jul 01 16:28:00.934015 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 16:28:00.934015 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application created_value = creator( Jul 01 16:28:00.934973 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 01 16:28:00.934973 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 01 16:28:00.934973 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 16:28:00.934973 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Jul 01 16:28:00.934973 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 382/787] 10.222.0.57 () {56 vars in 1108 bytes} [Fri Jul 1 16:28:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 301 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 16:28:00.940041 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdbf8615-0649-4898-a264-673c046223c8 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:28:01.074214 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f72daa-7b56-4baa-bfab-18d4235a1e68 None None] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:28:01.096271 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=100 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:28:01.147714 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdbf8615-0649-4898-a264-673c046223c8 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:28:01.161068 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29733690-ab38-417c-a905-e767d8aaca50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:28:01.163234 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29733690-ab38-417c-a905-e767d8aaca50 None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:28:01.163542 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29733690-ab38-417c-a905-e767d8aaca50 None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:28:01.163821 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29733690-ab38-417c-a905-e767d8aaca50 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:28:01.186618 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29733690-ab38-417c-a905-e767d8aaca50 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 01 16:28:01.203612 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application [None req-29733690-ab38-417c-a905-e767d8aaca50 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 01 16:28:01.203612 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 16:28:01.203612 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 16:28:01.203612 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 16:28:01.203612 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 16:28:01.203612 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 16:28:01.203612 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 16:28:01.203612 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 16:28:01.203612 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 16:28:01.203612 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 16:28:01.203612 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 16:28:01.203612 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 16:28:01.203612 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 01 16:28:01.203612 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 01 16:28:01.203612 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 01 16:28:01.203612 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 01 16:28:01.204627 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 01 16:28:01.204627 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 01 16:28:01.204627 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 01 16:28:01.204627 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 01 16:28:01.204627 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 16:28:01.204627 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 16:28:01.204627 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 16:28:01.204627 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 16:28:01.204627 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 16:28:01.204627 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 16:28:01.204627 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 16:28:01.204627 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application with Lock( Jul 01 16:28:01.204627 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 16:28:01.204627 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self._enter() Jul 01 16:28:01.204627 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 16:28:01.204627 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 16:28:01.205465 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 16:28:01.205465 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.creator() Jul 01 16:28:01.205465 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 16:28:01.205465 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application created_value = creator( Jul 01 16:28:01.205465 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 01 16:28:01.205465 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 01 16:28:01.205465 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 01 16:28:01.205465 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 01 16:28:01.205465 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 01 16:28:01.205465 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 01 16:28:01.205465 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 01 16:28:01.205465 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application Jul 01 16:28:01.205465 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 406/788] 10.222.0.57 () {56 vars in 1096 bytes} [Fri Jul 1 16:28:00 2022] GET /identity/v3/roles/admin => generated 83 bytes in 311 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 16:28:01.211738 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29733690-ab38-417c-a905-e767d8aaca50 None admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:28:01.230034 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0278d7a2-c007-47f4-8b17-3faf18f38688 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:28:01.230739 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0278d7a2-c007-47f4-8b17-3faf18f38688 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:28:01.231025 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0278d7a2-c007-47f4-8b17-3faf18f38688 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:28:01.231307 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0278d7a2-c007-47f4-8b17-3faf18f38688 None admin] PATH_INFO: `/v3/domains` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:28:01.232101 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0278d7a2-c007-47f4-8b17-3faf18f38688 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:28:01.232462 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0278d7a2-c007-47f4-8b17-3faf18f38688 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:28:01.233851 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0278d7a2-c007-47f4-8b17-3faf18f38688 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:28:01.248553 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 383/789] 10.222.0.57 () {56 vars in 1109 bytes} [Fri Jul 1 16:28:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 309 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:28:01.256044 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0278d7a2-c007-47f4-8b17-3faf18f38688 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:28:01.304338 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:28:01.395634 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29733690-ab38-417c-a905-e767d8aaca50 None admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:28:01.428770 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0278d7a2-c007-47f4-8b17-3faf18f38688 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:28:01.436511 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c518ae-1b75-49aa-b8e9-042a01b18440 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:28:01.437853 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c518ae-1b75-49aa-b8e9-042a01b18440 None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:28:01.438375 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c518ae-1b75-49aa-b8e9-042a01b18440 None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:28:01.438983 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c518ae-1b75-49aa-b8e9-042a01b18440 None admin] PATH_INFO: `/v3/roles` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:28:01.440839 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c518ae-1b75-49aa-b8e9-042a01b18440 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80312) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:28:01.441559 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c518ae-1b75-49aa-b8e9-042a01b18440 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:28:01.446448 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c518ae-1b75-49aa-b8e9-042a01b18440 None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:28:01.459339 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 407/790] 10.222.0.57 () {56 vars in 1099 bytes} [Fri Jul 1 16:28:01 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:28:01.461609 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcd12d9-53b2-4414-a24e-2ecf5893ac13 None None] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:28:01.462161 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcd12d9-53b2-4414-a24e-2ecf5893ac13 None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:28:01.462677 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcd12d9-53b2-4414-a24e-2ecf5893ac13 None None] PATH_INFO: `/` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:28:01.470917 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 408/791] 10.222.0.57 () {54 vars in 926 bytes} [Fri Jul 1 16:28:01 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 16:28:01.472502 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adcd12d9-53b2-4414-a24e-2ecf5893ac13 None None] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:28:01.480817 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d0960b-e4db-471f-9674-2f8c5c7e2086 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:28:01.482850 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d0960b-e4db-471f-9674-2f8c5c7e2086 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:28:01.482850 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d0960b-e4db-471f-9674-2f8c5c7e2086 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:28:01.482850 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d0960b-e4db-471f-9674-2f8c5c7e2086 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:28:01.491964 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d0960b-e4db-471f-9674-2f8c5c7e2086 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:28:01.493540 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d0960b-e4db-471f-9674-2f8c5c7e2086 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:28:01.512337 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application [None req-e1d0960b-e4db-471f-9674-2f8c5c7e2086 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 01 16:28:01.512337 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 16:28:01.512337 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 16:28:01.512337 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 16:28:01.512337 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 16:28:01.512337 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 16:28:01.512337 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 16:28:01.512337 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 16:28:01.512337 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 16:28:01.512337 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 16:28:01.512337 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 16:28:01.512337 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 16:28:01.512337 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 01 16:28:01.512337 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 01 16:28:01.512337 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 01 16:28:01.512337 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 01 16:28:01.513565 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 16:28:01.513565 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 16:28:01.513565 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 01 16:28:01.513565 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 01 16:28:01.513565 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 01 16:28:01.513565 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 01 16:28:01.513565 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 16:28:01.513565 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 16:28:01.513565 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 16:28:01.513565 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 16:28:01.513565 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 16:28:01.513565 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application with Lock( Jul 01 16:28:01.513565 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 16:28:01.513565 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._enter() Jul 01 16:28:01.513565 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 16:28:01.513565 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 16:28:01.513565 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 16:28:01.513565 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.creator() Jul 01 16:28:01.514959 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 16:28:01.514959 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application created_value = creator( Jul 01 16:28:01.514959 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 01 16:28:01.514959 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 01 16:28:01.514959 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 01 16:28:01.514959 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 01 16:28:01.514959 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 01 16:28:01.514959 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 01 16:28:01.514959 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 01 16:28:01.514959 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Jul 01 16:28:01.516257 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 384/792] 10.222.0.57 () {56 vars in 1117 bytes} [Fri Jul 1 16:28:01 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 16:28:01.517749 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=100 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:28:01.518250 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f27fba-e181-415a-a395-54712c90ba69 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:28:01.518706 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f27fba-e181-415a-a395-54712c90ba69 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:28:01.519123 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f27fba-e181-415a-a395-54712c90ba69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:28:01.538633 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-21f27fba-e181-415a-a395-54712c90ba69 None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:28:01.540269 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-21f27fba-e181-415a-a395-54712c90ba69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:28:01.626304 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 385/793] 10.222.0.57 () {58 vars in 1023 bytes} [Fri Jul 1 16:28:01 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 01 16:28:01.627054 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f27fba-e181-415a-a395-54712c90ba69 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:28:01.632627 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adcd12d9-53b2-4414-a24e-2ecf5893ac13 None None] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:28:01.644182 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:28:01.689460 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf9dcf5-7116-4858-a4e7-bf8df6828996 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:28:01.690206 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf9dcf5-7116-4858-a4e7-bf8df6828996 None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:28:01.690537 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf9dcf5-7116-4858-a4e7-bf8df6828996 None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:28:01.690814 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf9dcf5-7116-4858-a4e7-bf8df6828996 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:28:01.707419 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf9dcf5-7116-4858-a4e7-bf8df6828996 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:28:01.710260 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf9dcf5-7116-4858-a4e7-bf8df6828996 None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:28:01.733690 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application [None req-0bf9dcf5-7116-4858-a4e7-bf8df6828996 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 16:28:01.733690 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 16:28:01.733690 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 16:28:01.733690 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 16:28:01.733690 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 16:28:01.733690 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 16:28:01.733690 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 16:28:01.733690 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 16:28:01.733690 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 16:28:01.733690 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 16:28:01.733690 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 16:28:01.733690 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 16:28:01.733690 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 01 16:28:01.733690 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 01 16:28:01.733690 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 01 16:28:01.734853 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 01 16:28:01.734853 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 16:28:01.734853 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 16:28:01.734853 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 16:28:01.734853 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 16:28:01.734853 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 16:28:01.734853 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 16:28:01.734853 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 16:28:01.734853 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application with Lock( Jul 01 16:28:01.734853 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 16:28:01.734853 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self._enter() Jul 01 16:28:01.734853 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 16:28:01.734853 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 16:28:01.734853 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 16:28:01.734853 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.creator() Jul 01 16:28:01.734853 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 16:28:01.734853 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application created_value = creator( Jul 01 16:28:01.735945 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 01 16:28:01.735945 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 01 16:28:01.735945 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 16:28:01.735945 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application Jul 01 16:28:01.742019 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 409/794] 10.222.0.57 () {56 vars in 1108 bytes} [Fri Jul 1 16:28:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 266 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 16:28:01.742019 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf9dcf5-7116-4858-a4e7-bf8df6828996 None admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:28:01.765767 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f27fba-e181-415a-a395-54712c90ba69 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:28:01.806041 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d7b8fe-14c1-4fd1-88a7-596ac6104119 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:28:01.806978 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d7b8fe-14c1-4fd1-88a7-596ac6104119 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:28:01.807365 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d7b8fe-14c1-4fd1-88a7-596ac6104119 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:28:01.807744 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d7b8fe-14c1-4fd1-88a7-596ac6104119 None admin] PATH_INFO: `/v3/users` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:28:01.809140 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d7b8fe-14c1-4fd1-88a7-596ac6104119 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:28:01.809558 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d7b8fe-14c1-4fd1-88a7-596ac6104119 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:28:01.811448 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d7b8fe-14c1-4fd1-88a7-596ac6104119 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:28:01.843664 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 386/795] 10.222.0.57 () {56 vars in 1149 bytes} [Fri Jul 1 16:28:01 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:28:01.844815 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d7b8fe-14c1-4fd1-88a7-596ac6104119 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:28:01.857793 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=100 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:28:01.883512 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf9dcf5-7116-4858-a4e7-bf8df6828996 None admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:28:01.925619 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4353dc13-bdf1-4474-91e4-4b288a9f4afa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:28:01.927215 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4353dc13-bdf1-4474-91e4-4b288a9f4afa None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:28:01.928157 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4353dc13-bdf1-4474-91e4-4b288a9f4afa None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:28:01.928852 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4353dc13-bdf1-4474-91e4-4b288a9f4afa None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:28:01.942414 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4353dc13-bdf1-4474-91e4-4b288a9f4afa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 01 16:28:01.942583 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:28:01.953123 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application [None req-4353dc13-bdf1-4474-91e4-4b288a9f4afa None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 01 16:28:01.953123 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 16:28:01.953123 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 16:28:01.953123 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 16:28:01.953123 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 16:28:01.953123 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 16:28:01.953123 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 16:28:01.953123 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 16:28:01.953123 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 16:28:01.953123 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 16:28:01.953123 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 16:28:01.953123 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 16:28:01.953123 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 01 16:28:01.953123 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 01 16:28:01.953123 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 01 16:28:01.953123 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 01 16:28:01.954646 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 01 16:28:01.954646 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 01 16:28:01.954646 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 01 16:28:01.954646 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 01 16:28:01.954646 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 16:28:01.954646 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 16:28:01.954646 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 16:28:01.954646 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 16:28:01.954646 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 16:28:01.954646 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 16:28:01.954646 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 16:28:01.954646 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application with Lock( Jul 01 16:28:01.954646 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 16:28:01.954646 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self._enter() Jul 01 16:28:01.954646 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 16:28:01.954646 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 16:28:01.955710 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 16:28:01.955710 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.creator() Jul 01 16:28:01.955710 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 16:28:01.955710 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application created_value = creator( Jul 01 16:28:01.955710 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 01 16:28:01.955710 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 01 16:28:01.955710 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 01 16:28:01.955710 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 01 16:28:01.955710 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 01 16:28:01.955710 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 01 16:28:01.955710 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 01 16:28:01.955710 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application Jul 01 16:28:01.958929 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 410/796] 10.222.0.57 () {56 vars in 1096 bytes} [Fri Jul 1 16:28:01 2022] GET /identity/v3/roles/admin => generated 83 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 16:28:01.960170 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4353dc13-bdf1-4474-91e4-4b288a9f4afa None admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:28:01.981455 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d7b8fe-14c1-4fd1-88a7-596ac6104119 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:28:02.020523 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215420f3-4597-4126-b722-f6dbccf1bd18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:28:02.021352 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215420f3-4597-4126-b722-f6dbccf1bd18 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:28:02.021733 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215420f3-4597-4126-b722-f6dbccf1bd18 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:28:02.022153 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215420f3-4597-4126-b722-f6dbccf1bd18 None admin] PATH_INFO: `/v3/domains` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:28:02.022979 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-215420f3-4597-4126-b722-f6dbccf1bd18 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:28:02.023388 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-215420f3-4597-4126-b722-f6dbccf1bd18 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:28:02.025140 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-215420f3-4597-4126-b722-f6dbccf1bd18 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:28:02.044923 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 387/797] 10.222.0.57 () {56 vars in 1109 bytes} [Fri Jul 1 16:28:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:28:02.044923 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215420f3-4597-4126-b722-f6dbccf1bd18 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:28:02.095003 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4353dc13-bdf1-4474-91e4-4b288a9f4afa None admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:28:02.141089 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052952ca-dc85-4a80-af63-6dc033e1e462 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:28:02.142385 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052952ca-dc85-4a80-af63-6dc033e1e462 None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:28:02.143181 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052952ca-dc85-4a80-af63-6dc033e1e462 None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:28:02.143615 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052952ca-dc85-4a80-af63-6dc033e1e462 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:28:02.160051 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-052952ca-dc85-4a80-af63-6dc033e1e462 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:28:02.162562 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-052952ca-dc85-4a80-af63-6dc033e1e462 None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:28:02.165649 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=100 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:28:02.187441 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application [None req-052952ca-dc85-4a80-af63-6dc033e1e462 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 16:28:02.187441 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 16:28:02.187441 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 16:28:02.187441 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 16:28:02.187441 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 16:28:02.187441 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 16:28:02.187441 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 16:28:02.187441 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 16:28:02.187441 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 16:28:02.187441 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 16:28:02.187441 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 16:28:02.187441 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 16:28:02.187441 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 01 16:28:02.187441 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 01 16:28:02.187441 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 01 16:28:02.188610 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 01 16:28:02.188610 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 16:28:02.188610 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 16:28:02.188610 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 16:28:02.188610 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 16:28:02.188610 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 16:28:02.188610 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 16:28:02.188610 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 16:28:02.188610 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application with Lock( Jul 01 16:28:02.188610 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 16:28:02.188610 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self._enter() Jul 01 16:28:02.188610 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 16:28:02.188610 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 16:28:02.188610 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 16:28:02.188610 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.creator() Jul 01 16:28:02.188610 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 16:28:02.188610 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application created_value = creator( Jul 01 16:28:02.189718 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 01 16:28:02.189718 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 01 16:28:02.189718 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 16:28:02.189718 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application Jul 01 16:28:02.192069 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 411/798] 10.222.0.57 () {56 vars in 1108 bytes} [Fri Jul 1 16:28:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 16:28:02.193358 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052952ca-dc85-4a80-af63-6dc033e1e462 None admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:28:02.204344 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215420f3-4597-4126-b722-f6dbccf1bd18 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:28:02.239485 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:28:02.247505 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ed3f19-6fac-4df0-a68d-cfce87c46363 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:28:02.248740 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ed3f19-6fac-4df0-a68d-cfce87c46363 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:28:02.249199 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ed3f19-6fac-4df0-a68d-cfce87c46363 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:28:02.249619 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ed3f19-6fac-4df0-a68d-cfce87c46363 None admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:28:02.250822 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ed3f19-6fac-4df0-a68d-cfce87c46363 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:28:02.251740 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ed3f19-6fac-4df0-a68d-cfce87c46363 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:28:02.253597 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ed3f19-6fac-4df0-a68d-cfce87c46363 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:28:02.267137 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 388/799] 10.222.0.57 () {56 vars in 1099 bytes} [Fri Jul 1 16:28:02 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:28:02.268498 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ed3f19-6fac-4df0-a68d-cfce87c46363 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:28:02.348398 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052952ca-dc85-4a80-af63-6dc033e1e462 None admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:28:02.390200 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ad525e-524e-4ec3-a2ff-0555bfc9f6ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:28:02.391276 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ad525e-524e-4ec3-a2ff-0555bfc9f6ee None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:28:02.391724 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ad525e-524e-4ec3-a2ff-0555bfc9f6ee None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:28:02.392131 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ad525e-524e-4ec3-a2ff-0555bfc9f6ee None admin] PATH_INFO: `/v3/users/nova` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:28:02.403743 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ad525e-524e-4ec3-a2ff-0555bfc9f6ee None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:28:02.405729 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ad525e-524e-4ec3-a2ff-0555bfc9f6ee None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:28:02.417410 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ed3f19-6fac-4df0-a68d-cfce87c46363 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:28:02.423965 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application [None req-f6ad525e-524e-4ec3-a2ff-0555bfc9f6ee None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 01 16:28:02.423965 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 16:28:02.423965 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 16:28:02.423965 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 16:28:02.423965 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 16:28:02.423965 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 16:28:02.423965 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 16:28:02.423965 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 16:28:02.423965 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 16:28:02.423965 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 16:28:02.423965 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 16:28:02.423965 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 16:28:02.423965 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 01 16:28:02.423965 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 01 16:28:02.423965 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 01 16:28:02.423965 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 01 16:28:02.424949 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 16:28:02.424949 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 16:28:02.424949 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 01 16:28:02.424949 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 01 16:28:02.424949 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 01 16:28:02.424949 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 01 16:28:02.424949 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 16:28:02.424949 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 16:28:02.424949 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 16:28:02.424949 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 16:28:02.424949 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 16:28:02.424949 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application with Lock( Jul 01 16:28:02.424949 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 16:28:02.424949 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self._enter() Jul 01 16:28:02.424949 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 16:28:02.424949 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 16:28:02.424949 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 16:28:02.424949 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.creator() Jul 01 16:28:02.425935 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 16:28:02.425935 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application created_value = creator( Jul 01 16:28:02.425935 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 01 16:28:02.425935 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 01 16:28:02.425935 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 01 16:28:02.425935 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 01 16:28:02.425935 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 01 16:28:02.425935 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 01 16:28:02.425935 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 01 16:28:02.425935 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application Jul 01 16:28:02.425935 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 412/800] 10.222.0.57 () {56 vars in 1093 bytes} [Fri Jul 1 16:28:02 2022] GET /identity/v3/users/nova => generated 82 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 16:28:02.428349 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ad525e-524e-4ec3-a2ff-0555bfc9f6ee None admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:28:02.465602 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b08863-1034-4569-a59a-ad59c7d35691 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:28:02.466186 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b08863-1034-4569-a59a-ad59c7d35691 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:28:02.466511 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b08863-1034-4569-a59a-ad59c7d35691 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:28:02.466794 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b08863-1034-4569-a59a-ad59c7d35691 None admin] PATH_INFO: `/v3/domains` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:28:02.467516 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b08863-1034-4569-a59a-ad59c7d35691 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:28:02.467829 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b08863-1034-4569-a59a-ad59c7d35691 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:28:02.469202 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b08863-1034-4569-a59a-ad59c7d35691 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:28:02.482994 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 389/801] 10.222.0.57 () {56 vars in 1109 bytes} [Fri Jul 1 16:28:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:28:02.488142 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a882d308-a3b4-4366-9343-25f1bdbbe64e None None] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:28:02.488733 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a882d308-a3b4-4366-9343-25f1bdbbe64e None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:28:02.489042 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a882d308-a3b4-4366-9343-25f1bdbbe64e None None] PATH_INFO: `/` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:28:02.491032 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 390/802] 10.222.0.57 () {54 vars in 926 bytes} [Fri Jul 1 16:28:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 16:28:02.491593 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a882d308-a3b4-4366-9343-25f1bdbbe64e None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:28:02.498579 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=100 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:28:02.556824 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ad525e-524e-4ec3-a2ff-0555bfc9f6ee None admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:28:02.563464 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:28:02.596501 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b235d28e-7f77-4a48-bb67-04f37eb3bba5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:28:02.597507 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b235d28e-7f77-4a48-bb67-04f37eb3bba5 None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:28:02.597848 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b235d28e-7f77-4a48-bb67-04f37eb3bba5 None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:28:02.598133 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b235d28e-7f77-4a48-bb67-04f37eb3bba5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:28:02.611526 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b235d28e-7f77-4a48-bb67-04f37eb3bba5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:28:02.612988 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b235d28e-7f77-4a48-bb67-04f37eb3bba5 None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:28:02.629514 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a882d308-a3b4-4366-9343-25f1bdbbe64e None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:28:02.636314 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application [None req-b235d28e-7f77-4a48-bb67-04f37eb3bba5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 16:28:02.636314 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 16:28:02.636314 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 16:28:02.636314 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 16:28:02.636314 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 16:28:02.636314 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 16:28:02.636314 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 16:28:02.636314 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 16:28:02.636314 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 16:28:02.636314 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 16:28:02.636314 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 16:28:02.636314 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 16:28:02.636314 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 01 16:28:02.636314 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 01 16:28:02.636314 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 01 16:28:02.637691 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 01 16:28:02.637691 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 16:28:02.637691 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 16:28:02.637691 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 16:28:02.637691 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 16:28:02.637691 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 16:28:02.637691 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 16:28:02.637691 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 16:28:02.637691 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application with Lock( Jul 01 16:28:02.637691 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 16:28:02.637691 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self._enter() Jul 01 16:28:02.637691 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 16:28:02.637691 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 16:28:02.637691 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 16:28:02.637691 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.creator() Jul 01 16:28:02.637691 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 16:28:02.637691 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application created_value = creator( Jul 01 16:28:02.639909 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 01 16:28:02.639909 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 01 16:28:02.639909 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 16:28:02.639909 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application Jul 01 16:28:02.642742 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 413/803] 10.222.0.57 () {56 vars in 1108 bytes} [Fri Jul 1 16:28:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 16:28:02.643360 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b235d28e-7f77-4a48-bb67-04f37eb3bba5 None admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:28:02.673909 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667d19aa-a0b2-4777-9215-9e98c3b4948f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:28:02.674604 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667d19aa-a0b2-4777-9215-9e98c3b4948f None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:28:02.675004 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667d19aa-a0b2-4777-9215-9e98c3b4948f None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:28:02.675284 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667d19aa-a0b2-4777-9215-9e98c3b4948f None admin] PATH_INFO: `/v3/users` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:28:02.676285 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-667d19aa-a0b2-4777-9215-9e98c3b4948f None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:28:02.676597 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-667d19aa-a0b2-4777-9215-9e98c3b4948f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:28:02.678928 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-667d19aa-a0b2-4777-9215-9e98c3b4948f None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:28:02.698402 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 391/804] 10.222.0.57 () {56 vars in 1133 bytes} [Fri Jul 1 16:28:02 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:28:02.699574 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5713a00-3561-444d-b723-e897ec1907e0 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:28:02.699865 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5713a00-3561-444d-b723-e897ec1907e0 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:28:02.700150 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5713a00-3561-444d-b723-e897ec1907e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:28:02.719764 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-b5713a00-3561-444d-b723-e897ec1907e0 None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:28:02.720877 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-b5713a00-3561-444d-b723-e897ec1907e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:28:02.754776 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b235d28e-7f77-4a48-bb67-04f37eb3bba5 None admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:28:02.762684 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=100 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:28:02.792423 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83a74f6-abb1-46e0-962e-593a13e99a44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:28:02.793143 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 392/805] 10.222.0.57 () {58 vars in 1023 bytes} [Fri Jul 1 16:28:02 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 01 16:28:02.793280 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83a74f6-abb1-46e0-962e-593a13e99a44 None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:28:02.793485 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83a74f6-abb1-46e0-962e-593a13e99a44 None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:28:02.793761 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83a74f6-abb1-46e0-962e-593a13e99a44 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:28:02.793981 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5713a00-3561-444d-b723-e897ec1907e0 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:28:02.800894 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a83a74f6-abb1-46e0-962e-593a13e99a44 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:28:02.802539 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a83a74f6-abb1-46e0-962e-593a13e99a44 None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:28:02.817321 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application [None req-a83a74f6-abb1-46e0-962e-593a13e99a44 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 01 16:28:02.817321 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 16:28:02.817321 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 16:28:02.817321 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 16:28:02.817321 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 16:28:02.817321 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 16:28:02.817321 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 16:28:02.817321 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 16:28:02.817321 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 16:28:02.817321 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 16:28:02.817321 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 16:28:02.817321 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 16:28:02.817321 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 01 16:28:02.817321 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 01 16:28:02.817321 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 01 16:28:02.819183 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 01 16:28:02.819183 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 16:28:02.819183 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 16:28:02.819183 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 16:28:02.819183 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 16:28:02.819183 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 16:28:02.819183 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 16:28:02.819183 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 16:28:02.819183 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application with Lock( Jul 01 16:28:02.819183 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 16:28:02.819183 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self._enter() Jul 01 16:28:02.819183 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 16:28:02.819183 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 16:28:02.819183 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 16:28:02.819183 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.creator() Jul 01 16:28:02.819183 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 16:28:02.819183 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application created_value = creator( Jul 01 16:28:02.820586 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 01 16:28:02.820586 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 01 16:28:02.820586 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 01 16:28:02.820586 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 01 16:28:02.820586 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 01 16:28:02.820586 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 01 16:28:02.820586 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 01 16:28:02.820586 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application Jul 01 16:28:02.820586 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 414/806] 10.222.0.57 () {56 vars in 1111 bytes} [Fri Jul 1 16:28:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 16:28:02.820586 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753eb77e-5f4d-4048-a2c3-71b55e091bee None None] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:28:02.821432 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753eb77e-5f4d-4048-a2c3-71b55e091bee None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:28:02.821432 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753eb77e-5f4d-4048-a2c3-71b55e091bee None None] PATH_INFO: `/` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:28:02.822418 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 415/807] 10.222.0.57 () {54 vars in 926 bytes} [Fri Jul 1 16:28:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 16:28:02.826823 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753eb77e-5f4d-4048-a2c3-71b55e091bee None None] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:28:02.847078 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:28:02.926113 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5713a00-3561-444d-b723-e897ec1907e0 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:28:02.949276 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753eb77e-5f4d-4048-a2c3-71b55e091bee None None] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:28:02.962745 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5926cca4-ac89-403f-854f-68bbd7038d54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:28:02.963409 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5926cca4-ac89-403f-854f-68bbd7038d54 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:28:02.963688 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5926cca4-ac89-403f-854f-68bbd7038d54 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:28:02.963969 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5926cca4-ac89-403f-854f-68bbd7038d54 None admin] PATH_INFO: `/v3/domains` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:28:02.964692 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5926cca4-ac89-403f-854f-68bbd7038d54 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:28:02.964999 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5926cca4-ac89-403f-854f-68bbd7038d54 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:28:02.966371 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5926cca4-ac89-403f-854f-68bbd7038d54 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:28:02.978508 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 393/808] 10.222.0.57 () {56 vars in 1109 bytes} [Fri Jul 1 16:28:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:28:02.979563 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5926cca4-ac89-403f-854f-68bbd7038d54 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:28:02.987949 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a1427f-6824-48be-ac35-4653dd868158 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:28:02.988601 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a1427f-6824-48be-ac35-4653dd868158 None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:28:02.988878 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a1427f-6824-48be-ac35-4653dd868158 None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:28:02.989150 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a1427f-6824-48be-ac35-4653dd868158 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:28:03.024008 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=100 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:28:03.059779 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a1427f-6824-48be-ac35-4653dd868158 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:28:03.061232 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a1427f-6824-48be-ac35-4653dd868158 None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:28:03.084613 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5926cca4-ac89-403f-854f-68bbd7038d54 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:28:03.086150 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:28:03.120157 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e131383-77fe-437a-9fad-eb255dde70ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:28:03.120963 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e131383-77fe-437a-9fad-eb255dde70ae None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:28:03.121250 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e131383-77fe-437a-9fad-eb255dde70ae None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:28:03.121528 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e131383-77fe-437a-9fad-eb255dde70ae None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:28:03.122184 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e131383-77fe-437a-9fad-eb255dde70ae None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:28:03.123594 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e131383-77fe-437a-9fad-eb255dde70ae None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:28:03.135578 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application [None req-6e131383-77fe-437a-9fad-eb255dde70ae None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 01 16:28:03.135578 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 16:28:03.135578 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 01 16:28:03.135578 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 01 16:28:03.135578 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 01 16:28:03.135578 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 01 16:28:03.135578 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 01 16:28:03.135578 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 01 16:28:03.135578 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 01 16:28:03.135578 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Jul 01 16:28:03.135578 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 01 16:28:03.135578 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Jul 01 16:28:03.135578 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 16:28:03.135578 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 16:28:03.135578 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 16:28:03.135578 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 16:28:03.136621 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 16:28:03.136621 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 16:28:03.136621 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 16:28:03.136621 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 16:28:03.136621 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 16:28:03.136621 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 16:28:03.136621 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 16:28:03.136621 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 01 16:28:03.136621 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 01 16:28:03.136621 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 01 16:28:03.136621 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 01 16:28:03.136621 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 16:28:03.136621 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 16:28:03.136621 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 16:28:03.136621 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 16:28:03.136621 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 16:28:03.137566 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 16:28:03.137566 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 16:28:03.137566 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application with Lock( Jul 01 16:28:03.137566 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 16:28:03.137566 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._enter() Jul 01 16:28:03.137566 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 16:28:03.137566 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 16:28:03.137566 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 16:28:03.137566 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.creator() Jul 01 16:28:03.137566 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 16:28:03.137566 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application created_value = creator( Jul 01 16:28:03.137566 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 01 16:28:03.137566 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 01 16:28:03.137566 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 01 16:28:03.137566 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Jul 01 16:28:03.137566 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 394/809] 10.222.0.57 () {56 vars in 1129 bytes} [Fri Jul 1 16:28:02 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 16:28:03.138869 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e131383-77fe-437a-9fad-eb255dde70ae None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:28:03.166738 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 416/810] 10.222.0.57 () {58 vars in 1282 bytes} [Fri Jul 1 16:28:02 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 01 16:28:03.168145 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9126a667-3ba9-4c7c-b594-6f7a4ba1043b None None] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:28:03.168438 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9126a667-3ba9-4c7c-b594-6f7a4ba1043b None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:28:03.168715 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9126a667-3ba9-4c7c-b594-6f7a4ba1043b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:28:03.184570 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.auth.core [None req-9126a667-3ba9-4c7c-b594-6f7a4ba1043b None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80312) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:28:03.185758 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-9126a667-3ba9-4c7c-b594-6f7a4ba1043b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:28:03.254276 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=100 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:28:03.263883 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e131383-77fe-437a-9fad-eb255dde70ae None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:28:03.264610 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 417/811] 10.222.0.57 () {58 vars in 1023 bytes} [Fri Jul 1 16:28:03 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 01 16:28:03.265467 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9126a667-3ba9-4c7c-b594-6f7a4ba1043b None None] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:28:03.304000 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c8e0a0-2287-4538-8ea7-b5d93c0c4e5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:28:03.304663 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c8e0a0-2287-4538-8ea7-b5d93c0c4e5f None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:28:03.304944 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c8e0a0-2287-4538-8ea7-b5d93c0c4e5f None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:28:03.305222 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c8e0a0-2287-4538-8ea7-b5d93c0c4e5f None admin] PATH_INFO: `/v3/projects` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:28:03.306002 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c8e0a0-2287-4538-8ea7-b5d93c0c4e5f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:28:03.306313 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c8e0a0-2287-4538-8ea7-b5d93c0c4e5f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:28:03.307773 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c8e0a0-2287-4538-8ea7-b5d93c0c4e5f None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:28:03.320375 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 395/812] 10.222.0.57 () {56 vars in 1148 bytes} [Fri Jul 1 16:28:03 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:28:03.321193 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c8e0a0-2287-4538-8ea7-b5d93c0c4e5f None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:28:03.348777 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:28:03.378973 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9126a667-3ba9-4c7c-b594-6f7a4ba1043b None None] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:28:03.420021 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b984d298-fe64-4391-9b94-c6fc3554df80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:28:03.420900 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b984d298-fe64-4391-9b94-c6fc3554df80 None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:28:03.421298 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b984d298-fe64-4391-9b94-c6fc3554df80 None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:28:03.421662 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b984d298-fe64-4391-9b94-c6fc3554df80 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:28:03.429716 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b984d298-fe64-4391-9b94-c6fc3554df80 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:28:03.431197 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b984d298-fe64-4391-9b94-c6fc3554df80 None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:28:03.434825 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c8e0a0-2287-4538-8ea7-b5d93c0c4e5f None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:28:03.447135 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application [None req-b984d298-fe64-4391-9b94-c6fc3554df80 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 01 16:28:03.447135 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 16:28:03.447135 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 16:28:03.447135 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 16:28:03.447135 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 16:28:03.447135 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 16:28:03.447135 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 16:28:03.447135 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 16:28:03.447135 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 16:28:03.447135 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 16:28:03.447135 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 16:28:03.447135 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 16:28:03.447135 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 01 16:28:03.447135 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 01 16:28:03.447135 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 01 16:28:03.447135 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 01 16:28:03.448225 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 16:28:03.448225 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 16:28:03.448225 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 01 16:28:03.448225 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 01 16:28:03.448225 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 01 16:28:03.448225 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 01 16:28:03.448225 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 16:28:03.448225 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 16:28:03.448225 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 16:28:03.448225 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 16:28:03.448225 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 16:28:03.448225 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application with Lock( Jul 01 16:28:03.448225 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 16:28:03.448225 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self._enter() Jul 01 16:28:03.448225 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 16:28:03.448225 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 16:28:03.448225 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 16:28:03.448225 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.creator() Jul 01 16:28:03.449191 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 16:28:03.449191 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application created_value = creator( Jul 01 16:28:03.449191 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 01 16:28:03.449191 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 01 16:28:03.449191 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 01 16:28:03.449191 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 01 16:28:03.449191 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 01 16:28:03.449191 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 01 16:28:03.449191 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 01 16:28:03.449191 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application Jul 01 16:28:03.449191 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 418/813] 10.222.0.57 () {56 vars in 1096 bytes} [Fri Jul 1 16:28:03 2022] GET /identity/v3/users/swift => generated 83 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 16:28:03.449191 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b984d298-fe64-4391-9b94-c6fc3554df80 None admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:28:03.473076 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2665e747-3928-4207-9943-b5d06318e05c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:28:03.473739 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2665e747-3928-4207-9943-b5d06318e05c None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:28:03.474210 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2665e747-3928-4207-9943-b5d06318e05c None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:28:03.474447 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2665e747-3928-4207-9943-b5d06318e05c None admin] PATH_INFO: `/v3/services` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:28:03.475135 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2665e747-3928-4207-9943-b5d06318e05c None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:28:03.475442 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2665e747-3928-4207-9943-b5d06318e05c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:28:03.476830 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2665e747-3928-4207-9943-b5d06318e05c None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:28:03.481373 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 396/814] 10.222.0.57 () {56 vars in 1124 bytes} [Fri Jul 1 16:28:03 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:28:03.482160 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2665e747-3928-4207-9943-b5d06318e05c None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:28:03.515777 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=100 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:28:03.562311 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b984d298-fe64-4391-9b94-c6fc3554df80 None admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:28:03.581275 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:28:03.592309 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2665e747-3928-4207-9943-b5d06318e05c None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:28:03.602497 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4292b6c7-50f0-46cc-b669-26736de51afa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:28:03.603299 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4292b6c7-50f0-46cc-b669-26736de51afa None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:28:03.603577 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4292b6c7-50f0-46cc-b669-26736de51afa None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:28:03.603858 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4292b6c7-50f0-46cc-b669-26736de51afa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:28:03.615020 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4292b6c7-50f0-46cc-b669-26736de51afa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:28:03.616466 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4292b6c7-50f0-46cc-b669-26736de51afa None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:28:03.627808 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d4de74-f0f7-43d2-9a6a-89523bd1a5ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:28:03.628596 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d4de74-f0f7-43d2-9a6a-89523bd1a5ae None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:28:03.628885 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d4de74-f0f7-43d2-9a6a-89523bd1a5ae None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:28:03.629158 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d4de74-f0f7-43d2-9a6a-89523bd1a5ae None admin] PATH_INFO: `/v3/services/network` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:28:03.629814 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d4de74-f0f7-43d2-9a6a-89523bd1a5ae None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:28:03.631782 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d4de74-f0f7-43d2-9a6a-89523bd1a5ae None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:28:03.634627 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application [None req-4292b6c7-50f0-46cc-b669-26736de51afa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 16:28:03.634627 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 16:28:03.634627 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 16:28:03.634627 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 16:28:03.634627 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 16:28:03.634627 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 16:28:03.634627 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 16:28:03.634627 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 16:28:03.634627 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 16:28:03.634627 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 16:28:03.634627 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 16:28:03.634627 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 16:28:03.634627 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 01 16:28:03.634627 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 01 16:28:03.634627 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 01 16:28:03.635487 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 01 16:28:03.635487 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 16:28:03.635487 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 16:28:03.635487 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 16:28:03.635487 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 16:28:03.635487 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 16:28:03.635487 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 16:28:03.635487 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 16:28:03.635487 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application with Lock( Jul 01 16:28:03.635487 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 16:28:03.635487 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self._enter() Jul 01 16:28:03.635487 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 16:28:03.635487 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 16:28:03.635487 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 16:28:03.635487 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.creator() Jul 01 16:28:03.635487 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 16:28:03.635487 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application created_value = creator( Jul 01 16:28:03.636434 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 01 16:28:03.636434 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 01 16:28:03.636434 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 16:28:03.636434 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application Jul 01 16:28:03.640776 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 419/815] 10.222.0.57 () {56 vars in 1108 bytes} [Fri Jul 1 16:28:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 16:28:03.641659 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4292b6c7-50f0-46cc-b669-26736de51afa None admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:28:03.643768 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application [None req-85d4de74-f0f7-43d2-9a6a-89523bd1a5ae None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 01 16:28:03.643768 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 16:28:03.643768 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 01 16:28:03.643768 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 01 16:28:03.643768 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 01 16:28:03.643768 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 01 16:28:03.643768 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 01 16:28:03.643768 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 01 16:28:03.643768 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 01 16:28:03.643768 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Jul 01 16:28:03.643768 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 01 16:28:03.643768 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Jul 01 16:28:03.643768 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 16:28:03.643768 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 16:28:03.643768 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 16:28:03.643768 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 16:28:03.643768 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 16:28:03.646561 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 16:28:03.646561 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 16:28:03.646561 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 16:28:03.646561 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 16:28:03.646561 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 16:28:03.646561 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 16:28:03.646561 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 01 16:28:03.646561 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 01 16:28:03.646561 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 01 16:28:03.646561 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 01 16:28:03.646561 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 16:28:03.646561 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 16:28:03.646561 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 16:28:03.646561 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 16:28:03.646561 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 16:28:03.646561 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 16:28:03.647860 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 16:28:03.647860 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application with Lock( Jul 01 16:28:03.647860 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 16:28:03.647860 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._enter() Jul 01 16:28:03.647860 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 16:28:03.647860 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 16:28:03.647860 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 16:28:03.647860 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.creator() Jul 01 16:28:03.647860 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 16:28:03.647860 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application created_value = creator( Jul 01 16:28:03.647860 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 01 16:28:03.647860 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 01 16:28:03.647860 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 01 16:28:03.647860 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Jul 01 16:28:03.647860 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 397/816] 10.222.0.57 () {56 vars in 1111 bytes} [Fri Jul 1 16:28:03 2022] GET /identity/v3/services/network => generated 88 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 16:28:03.648837 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d4de74-f0f7-43d2-9a6a-89523bd1a5ae None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:28:03.754595 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4292b6c7-50f0-46cc-b669-26736de51afa None admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:28:03.765582 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=100 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:28:03.770424 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d4de74-f0f7-43d2-9a6a-89523bd1a5ae None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:28:03.791154 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fdefc3-63da-4d31-bb71-79974ae2f28e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:28:03.792041 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fdefc3-63da-4d31-bb71-79974ae2f28e None admin] REQUEST_METHOD: `PUT` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:28:03.792324 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fdefc3-63da-4d31-bb71-79974ae2f28e None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:28:03.792602 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fdefc3-63da-4d31-bb71-79974ae2f28e None admin] PATH_INFO: `/v3/projects/e0788e6725a84e57a4174284c1845755/users/c4de1b1eeaed4fc5ae7200b2c7a90957/roles/e8809b67c9dc471daa1c2bd42c9a2baf` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:28:03.806697 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbea0bd0-5dd0-47ad-8d0d-3678bb4436a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:28:03.807371 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbea0bd0-5dd0-47ad-8d0d-3678bb4436a0 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:28:03.807656 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbea0bd0-5dd0-47ad-8d0d-3678bb4436a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:28:03.807773 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49fdefc3-63da-4d31-bb71-79974ae2f28e None admin] RBAC: Authorizing `identity:create_grant(project_id=e0788e6725a84e57a4174284c1845755, user_id=c4de1b1eeaed4fc5ae7200b2c7a90957, role_id=e8809b67c9dc471daa1c2bd42c9a2baf)` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:28:03.807939 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbea0bd0-5dd0-47ad-8d0d-3678bb4436a0 None admin] PATH_INFO: `/v3/users` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:28:03.808780 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbea0bd0-5dd0-47ad-8d0d-3678bb4436a0 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:28:03.809093 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbea0bd0-5dd0-47ad-8d0d-3678bb4436a0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:28:03.809211 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49fdefc3-63da-4d31-bb71-79974ae2f28e None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:28:03.810572 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbea0bd0-5dd0-47ad-8d0d-3678bb4436a0 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:28:03.824823 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 420/817] 10.222.0.57 () {58 vars in 1439 bytes} [Fri Jul 1 16:28:03 2022] PUT /identity/v3/projects/e0788e6725a84e57a4174284c1845755/users/c4de1b1eeaed4fc5ae7200b2c7a90957/roles/e8809b67c9dc471daa1c2bd42c9a2baf => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 16:28:03.826185 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fdefc3-63da-4d31-bb71-79974ae2f28e None admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:28:03.846676 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 398/818] 10.222.0.57 () {56 vars in 1135 bytes} [Fri Jul 1 16:28:03 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:28:03.846676 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbea0bd0-5dd0-47ad-8d0d-3678bb4436a0 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:28:03.863644 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:28:03.983322 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbea0bd0-5dd0-47ad-8d0d-3678bb4436a0 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:28:03.996652 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fdefc3-63da-4d31-bb71-79974ae2f28e None admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:28:04.026819 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65dd5967-13b4-4bd7-8a52-a0abffeed8d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:28:04.027793 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65dd5967-13b4-4bd7-8a52-a0abffeed8d9 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:28:04.028514 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65dd5967-13b4-4bd7-8a52-a0abffeed8d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:28:04.028662 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65dd5967-13b4-4bd7-8a52-a0abffeed8d9 None admin] PATH_INFO: `/v3/domains` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:28:04.029283 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65dd5967-13b4-4bd7-8a52-a0abffeed8d9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:28:04.029587 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65dd5967-13b4-4bd7-8a52-a0abffeed8d9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:28:04.031258 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65dd5967-13b4-4bd7-8a52-a0abffeed8d9 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:28:04.044356 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87041d09-35d6-455e-9687-f6939fd625e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:28:04.044356 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87041d09-35d6-455e-9687-f6939fd625e8 None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:28:04.044680 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 399/819] 10.222.0.57 () {56 vars in 1109 bytes} [Fri Jul 1 16:28:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:28:04.044769 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87041d09-35d6-455e-9687-f6939fd625e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:28:04.045079 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65dd5967-13b4-4bd7-8a52-a0abffeed8d9 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:28:04.045172 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87041d09-35d6-455e-9687-f6939fd625e8 None admin] PATH_INFO: `/v3/services` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:28:04.046087 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87041d09-35d6-455e-9687-f6939fd625e8 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80312) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:28:04.046542 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87041d09-35d6-455e-9687-f6939fd625e8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:28:04.048017 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87041d09-35d6-455e-9687-f6939fd625e8 None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:28:04.058265 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 421/820] 10.222.0.57 () {56 vars in 1124 bytes} [Fri Jul 1 16:28:03 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:28:04.058265 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87041d09-35d6-455e-9687-f6939fd625e8 None admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:28:04.088577 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:28:04.127803 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:28:04.161844 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65dd5967-13b4-4bd7-8a52-a0abffeed8d9 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:28:04.182141 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87041d09-35d6-455e-9687-f6939fd625e8 None admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:28:04.200740 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e46317-6d0f-4165-9d3f-00e31b570f65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:28:04.201408 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e46317-6d0f-4165-9d3f-00e31b570f65 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:28:04.201706 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e46317-6d0f-4165-9d3f-00e31b570f65 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:28:04.201991 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e46317-6d0f-4165-9d3f-00e31b570f65 None admin] PATH_INFO: `/v3/services` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:28:04.202983 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e46317-6d0f-4165-9d3f-00e31b570f65 None admin] RBAC: Adding query filter params (name=network) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:28:04.203391 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e46317-6d0f-4165-9d3f-00e31b570f65 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:28:04.204704 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e46317-6d0f-4165-9d3f-00e31b570f65 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:28:04.210638 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 400/821] 10.222.0.57 () {56 vars in 1112 bytes} [Fri Jul 1 16:28:04 2022] GET /identity/v3/services?name=network => generated 124 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:28:04.211551 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e46317-6d0f-4165-9d3f-00e31b570f65 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:28:04.228106 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a31c32f-7602-481e-91f1-989969a2ee7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:28:04.229022 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a31c32f-7602-481e-91f1-989969a2ee7c None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:28:04.229292 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a31c32f-7602-481e-91f1-989969a2ee7c None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:28:04.229564 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a31c32f-7602-481e-91f1-989969a2ee7c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:28:04.241315 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a31c32f-7602-481e-91f1-989969a2ee7c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:28:04.243511 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a31c32f-7602-481e-91f1-989969a2ee7c None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:28:04.269506 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application [None req-9a31c32f-7602-481e-91f1-989969a2ee7c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 16:28:04.269506 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 16:28:04.269506 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 16:28:04.269506 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 16:28:04.269506 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 16:28:04.269506 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 16:28:04.269506 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 16:28:04.269506 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 16:28:04.269506 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 16:28:04.269506 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 16:28:04.269506 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 16:28:04.269506 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 16:28:04.269506 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 01 16:28:04.269506 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 01 16:28:04.269506 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 01 16:28:04.270800 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 01 16:28:04.270800 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 16:28:04.270800 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 16:28:04.270800 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 16:28:04.270800 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 16:28:04.270800 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 16:28:04.270800 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 16:28:04.270800 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 16:28:04.270800 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application with Lock( Jul 01 16:28:04.270800 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 16:28:04.270800 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self._enter() Jul 01 16:28:04.270800 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 16:28:04.270800 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 16:28:04.270800 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 16:28:04.270800 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.creator() Jul 01 16:28:04.270800 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 16:28:04.270800 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application created_value = creator( Jul 01 16:28:04.272579 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 01 16:28:04.272579 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 01 16:28:04.272579 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 16:28:04.272579 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application Jul 01 16:28:04.273482 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 422/822] 10.222.0.57 () {56 vars in 1108 bytes} [Fri Jul 1 16:28:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 16:28:04.274758 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a31c32f-7602-481e-91f1-989969a2ee7c None admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:28:04.326138 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e46317-6d0f-4165-9d3f-00e31b570f65 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:28:04.368754 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ce8631-aad6-4507-ab29-6305081e9226 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:28:04.369655 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ce8631-aad6-4507-ab29-6305081e9226 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:28:04.369940 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ce8631-aad6-4507-ab29-6305081e9226 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:28:04.370214 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ce8631-aad6-4507-ab29-6305081e9226 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:28:04.373088 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=100 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:28:04.373837 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:28:04.379592 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ce8631-aad6-4507-ab29-6305081e9226 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:28:04.382177 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ce8631-aad6-4507-ab29-6305081e9226 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:28:04.399376 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application [None req-37ce8631-aad6-4507-ab29-6305081e9226 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 01 16:28:04.399376 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 16:28:04.399376 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 16:28:04.399376 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 16:28:04.399376 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 16:28:04.399376 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 16:28:04.399376 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 16:28:04.399376 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 16:28:04.399376 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 16:28:04.399376 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 16:28:04.399376 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 16:28:04.399376 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 16:28:04.399376 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 01 16:28:04.399376 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 01 16:28:04.399376 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 01 16:28:04.406547 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 01 16:28:04.406547 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 16:28:04.406547 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 16:28:04.406547 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 16:28:04.406547 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 16:28:04.406547 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 16:28:04.406547 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 16:28:04.406547 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 16:28:04.406547 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application with Lock( Jul 01 16:28:04.406547 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 16:28:04.406547 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._enter() Jul 01 16:28:04.406547 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 16:28:04.406547 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 16:28:04.406547 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 16:28:04.406547 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.creator() Jul 01 16:28:04.406547 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 16:28:04.406547 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application created_value = creator( Jul 01 16:28:04.407511 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 01 16:28:04.407511 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 01 16:28:04.407511 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 01 16:28:04.407511 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 01 16:28:04.407511 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 01 16:28:04.407511 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 01 16:28:04.407511 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 01 16:28:04.407511 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Jul 01 16:28:04.407511 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 401/823] 10.222.0.57 () {56 vars in 1111 bytes} [Fri Jul 1 16:28:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 16:28:04.407511 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ce8631-aad6-4507-ab29-6305081e9226 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:28:04.414062 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a31c32f-7602-481e-91f1-989969a2ee7c None admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:28:04.454564 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd6e334-7985-40fa-ad7f-aaf41cb53429 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:28:04.455360 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd6e334-7985-40fa-ad7f-aaf41cb53429 None admin] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:28:04.455742 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd6e334-7985-40fa-ad7f-aaf41cb53429 None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:28:04.456115 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd6e334-7985-40fa-ad7f-aaf41cb53429 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:28:04.456949 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd6e334-7985-40fa-ad7f-aaf41cb53429 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:28:04.458612 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd6e334-7985-40fa-ad7f-aaf41cb53429 None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:28:04.481961 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 423/824] 10.222.0.57 () {60 vars in 1144 bytes} [Fri Jul 1 16:28:04 2022] POST /identity/v3/endpoints => generated 346 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 16:28:04.483070 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd6e334-7985-40fa-ad7f-aaf41cb53429 None admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:28:04.538399 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ce8631-aad6-4507-ab29-6305081e9226 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:28:04.592497 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8580a1b2-767f-4cd1-95e1-895964a1dfc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:28:04.593574 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8580a1b2-767f-4cd1-95e1-895964a1dfc8 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:28:04.594111 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8580a1b2-767f-4cd1-95e1-895964a1dfc8 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:28:04.594762 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8580a1b2-767f-4cd1-95e1-895964a1dfc8 None admin] PATH_INFO: `/v3/services` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:28:04.595786 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8580a1b2-767f-4cd1-95e1-895964a1dfc8 None admin] RBAC: Adding query filter params (type=network) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:28:04.596259 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8580a1b2-767f-4cd1-95e1-895964a1dfc8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:28:04.597980 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8580a1b2-767f-4cd1-95e1-895964a1dfc8 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:28:04.604883 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 402/825] 10.222.0.57 () {56 vars in 1112 bytes} [Fri Jul 1 16:28:04 2022] GET /identity/v3/services?type=network => generated 350 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:28:04.606276 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8580a1b2-767f-4cd1-95e1-895964a1dfc8 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:28:04.616078 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd6e334-7985-40fa-ad7f-aaf41cb53429 None admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:28:04.663833 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:28:04.674109 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f108cb2-aafe-4ffb-ac73-fd242c61f901 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:28:04.675627 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f108cb2-aafe-4ffb-ac73-fd242c61f901 None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:28:04.676386 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f108cb2-aafe-4ffb-ac73-fd242c61f901 None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:28:04.677431 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f108cb2-aafe-4ffb-ac73-fd242c61f901 None admin] PATH_INFO: `/v3/domains` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:28:04.678755 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f108cb2-aafe-4ffb-ac73-fd242c61f901 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80312) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:28:04.679226 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f108cb2-aafe-4ffb-ac73-fd242c61f901 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:28:04.682088 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f108cb2-aafe-4ffb-ac73-fd242c61f901 None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:28:04.691765 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:28:04.700623 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 424/826] 10.222.0.57 () {56 vars in 1109 bytes} [Fri Jul 1 16:28:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:28:04.700623 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f108cb2-aafe-4ffb-ac73-fd242c61f901 None admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:28:04.760583 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8580a1b2-767f-4cd1-95e1-895964a1dfc8 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:28:04.803202 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a4ea4f-1bd3-427d-aa48-ad39989e8cf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:28:04.804373 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a4ea4f-1bd3-427d-aa48-ad39989e8cf3 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:28:04.804858 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a4ea4f-1bd3-427d-aa48-ad39989e8cf3 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:28:04.805345 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a4ea4f-1bd3-427d-aa48-ad39989e8cf3 None admin] PATH_INFO: `/v3/projects` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:28:04.806441 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a4ea4f-1bd3-427d-aa48-ad39989e8cf3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:28:04.806951 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a4ea4f-1bd3-427d-aa48-ad39989e8cf3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:28:04.808710 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a4ea4f-1bd3-427d-aa48-ad39989e8cf3 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:28:04.824191 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 403/827] 10.222.0.57 () {56 vars in 1148 bytes} [Fri Jul 1 16:28:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:28:04.825474 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a4ea4f-1bd3-427d-aa48-ad39989e8cf3 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:28:04.852545 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f108cb2-aafe-4ffb-ac73-fd242c61f901 None admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:28:04.893951 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e81ac3-c87d-4ec2-b1a2-ca0a597024f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:28:04.894744 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e81ac3-c87d-4ec2-b1a2-ca0a597024f5 None admin] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:28:04.895032 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e81ac3-c87d-4ec2-b1a2-ca0a597024f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:28:04.895313 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e81ac3-c87d-4ec2-b1a2-ca0a597024f5 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:28:04.896821 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e81ac3-c87d-4ec2-b1a2-ca0a597024f5 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:28:04.898372 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e81ac3-c87d-4ec2-b1a2-ca0a597024f5 None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:28:04.921254 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 425/828] 10.222.0.57 () {60 vars in 1144 bytes} [Fri Jul 1 16:28:04 2022] POST /identity/v3/endpoints => generated 327 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 16:28:04.922151 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e81ac3-c87d-4ec2-b1a2-ca0a597024f5 None admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:28:04.941319 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a4ea4f-1bd3-427d-aa48-ad39989e8cf3 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:28:04.970606 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:28:05.004405 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db177906-60d1-4bb5-a9c9-88e2bdf0dea9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:28:05.005870 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db177906-60d1-4bb5-a9c9-88e2bdf0dea9 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:28:05.006611 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db177906-60d1-4bb5-a9c9-88e2bdf0dea9 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:28:05.007114 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db177906-60d1-4bb5-a9c9-88e2bdf0dea9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:28:05.011367 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:28:05.018656 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db177906-60d1-4bb5-a9c9-88e2bdf0dea9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:28:05.020621 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db177906-60d1-4bb5-a9c9-88e2bdf0dea9 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:28:05.038064 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application [None req-db177906-60d1-4bb5-a9c9-88e2bdf0dea9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 01 16:28:05.038064 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 16:28:05.038064 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 16:28:05.038064 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 16:28:05.038064 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 16:28:05.038064 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 16:28:05.038064 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 16:28:05.038064 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 16:28:05.038064 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 16:28:05.038064 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 16:28:05.038064 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 16:28:05.038064 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 16:28:05.038064 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 01 16:28:05.038064 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 01 16:28:05.038064 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 01 16:28:05.039093 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 01 16:28:05.039093 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 16:28:05.039093 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 16:28:05.039093 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 16:28:05.039093 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 16:28:05.039093 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 16:28:05.039093 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 16:28:05.039093 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 16:28:05.039093 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application with Lock( Jul 01 16:28:05.039093 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 16:28:05.039093 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._enter() Jul 01 16:28:05.039093 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 16:28:05.039093 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 16:28:05.039093 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 16:28:05.039093 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.creator() Jul 01 16:28:05.039093 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 16:28:05.039093 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application created_value = creator( Jul 01 16:28:05.040071 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 01 16:28:05.040071 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 01 16:28:05.040071 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 01 16:28:05.040071 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 01 16:28:05.040071 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 01 16:28:05.040071 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 01 16:28:05.040071 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 01 16:28:05.040071 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Jul 01 16:28:05.042821 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 404/829] 10.222.0.57 () {56 vars in 1111 bytes} [Fri Jul 1 16:28:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 16:28:05.049177 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db177906-60d1-4bb5-a9c9-88e2bdf0dea9 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:28:05.145021 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e81ac3-c87d-4ec2-b1a2-ca0a597024f5 None admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:28:05.187669 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81cc28d1-73d7-486b-895b-0c9582deacff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:28:05.188348 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cc28d1-73d7-486b-895b-0c9582deacff None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:28:05.188633 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cc28d1-73d7-486b-895b-0c9582deacff None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:28:05.188915 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cc28d1-73d7-486b-895b-0c9582deacff None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:28:05.189883 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81cc28d1-73d7-486b-895b-0c9582deacff None admin] RBAC: Adding query filter params (role.id=84ee739146934c1ab32c501ff8093644, scope.project.id=e0788e6725a84e57a4174284c1845755, user.id=7a50f745b31941b7a7265c0fc852f48e) {{(pid=80312) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:28:05.190186 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81cc28d1-73d7-486b-895b-0c9582deacff None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:28:05.191700 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81cc28d1-73d7-486b-895b-0c9582deacff None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:28:05.199206 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 426/830] 10.222.0.57 () {56 vars in 1374 bytes} [Fri Jul 1 16:28:04 2022] GET /identity/v3/role_assignments?user.id=7a50f745b31941b7a7265c0fc852f48e&scope.project.id=e0788e6725a84e57a4174284c1845755&role.id=84ee739146934c1ab32c501ff8093644 => generated 608 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:28:05.212830 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db177906-60d1-4bb5-a9c9-88e2bdf0dea9 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:28:05.253489 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e1a4d0-41dd-4e87-961d-216a17081730 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:28:05.254169 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e1a4d0-41dd-4e87-961d-216a17081730 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:28:05.254496 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e1a4d0-41dd-4e87-961d-216a17081730 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:28:05.254783 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e1a4d0-41dd-4e87-961d-216a17081730 None admin] PATH_INFO: `/v3/projects` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:28:05.255805 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e1a4d0-41dd-4e87-961d-216a17081730 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:28:05.256321 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e1a4d0-41dd-4e87-961d-216a17081730 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:28:05.257897 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e1a4d0-41dd-4e87-961d-216a17081730 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:28:05.273147 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 405/831] 10.222.0.57 () {56 vars in 1148 bytes} [Fri Jul 1 16:28:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 01 16:28:05.278966 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81cc28d1-73d7-486b-895b-0c9582deacff None admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:28:05.399825 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81cc28d1-73d7-486b-895b-0c9582deacff None admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:28:05.416034 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=100 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:28:05.437695 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655a9102-e71e-40ad-957a-66c238da99cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:28:05.438787 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655a9102-e71e-40ad-957a-66c238da99cb None admin] REQUEST_METHOD: `PUT` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:28:05.439135 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655a9102-e71e-40ad-957a-66c238da99cb None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:28:05.439478 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655a9102-e71e-40ad-957a-66c238da99cb None admin] PATH_INFO: `/v3/projects/e0788e6725a84e57a4174284c1845755/users/859fe410c2d2450fb803c6244839f27c/roles/84ee739146934c1ab32c501ff8093644` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:28:05.442048 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-655a9102-e71e-40ad-957a-66c238da99cb None admin] RBAC: Authorizing `identity:create_grant(project_id=e0788e6725a84e57a4174284c1845755, user_id=859fe410c2d2450fb803c6244839f27c, role_id=84ee739146934c1ab32c501ff8093644)` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:28:05.443967 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-655a9102-e71e-40ad-957a-66c238da99cb None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:28:05.461988 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 427/832] 10.222.0.57 () {58 vars in 1439 bytes} [Fri Jul 1 16:28:05 2022] PUT /identity/v3/projects/e0788e6725a84e57a4174284c1845755/users/859fe410c2d2450fb803c6244839f27c/roles/84ee739146934c1ab32c501ff8093644 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 01 16:28:06.094737 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8fb9be-8d4e-425c-bd9f-668aa56bc3f7 None None] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:28:06.095251 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8fb9be-8d4e-425c-bd9f-668aa56bc3f7 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:28:06.095327 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8fb9be-8d4e-425c-bd9f-668aa56bc3f7 None None] PATH_INFO: `/` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:28:06.096957 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 406/833] 10.222.0.57 () {54 vars in 926 bytes} [Fri Jul 1 16:28:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 16:28:06.103145 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5486f8-4c28-4dd2-94ec-a7bd4e6ce268 None None] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:28:06.103562 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5486f8-4c28-4dd2-94ec-a7bd4e6ce268 None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:28:06.103639 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5486f8-4c28-4dd2-94ec-a7bd4e6ce268 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:28:06.121681 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.auth.core [None req-cd5486f8-4c28-4dd2-94ec-a7bd4e6ce268 None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80312) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:28:06.122920 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-cd5486f8-4c28-4dd2-94ec-a7bd4e6ce268 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:28:06.197940 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 428/834] 10.222.0.57 () {58 vars in 1023 bytes} [Fri Jul 1 16:28:06 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 01 16:28:06.222292 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8fb9be-8d4e-425c-bd9f-668aa56bc3f7 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:28:06.254919 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:28:06.338773 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8fb9be-8d4e-425c-bd9f-668aa56bc3f7 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:28:06.374828 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ebc058-c7c7-491c-a438-23b483066ef6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:28:06.375545 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ebc058-c7c7-491c-a438-23b483066ef6 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:28:06.375819 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ebc058-c7c7-491c-a438-23b483066ef6 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:28:06.376093 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ebc058-c7c7-491c-a438-23b483066ef6 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:28:06.387668 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ebc058-c7c7-491c-a438-23b483066ef6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 01 16:28:06.395418 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application [None req-66ebc058-c7c7-491c-a438-23b483066ef6 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 01 16:28:06.395418 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 16:28:06.395418 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 16:28:06.395418 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 16:28:06.395418 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 16:28:06.395418 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 16:28:06.395418 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 16:28:06.395418 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 16:28:06.395418 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 16:28:06.395418 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 16:28:06.395418 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 16:28:06.395418 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 16:28:06.395418 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 01 16:28:06.395418 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 01 16:28:06.395418 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 01 16:28:06.395418 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 01 16:28:06.397114 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 01 16:28:06.397114 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 01 16:28:06.397114 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 01 16:28:06.397114 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 01 16:28:06.397114 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 16:28:06.397114 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 16:28:06.397114 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 16:28:06.397114 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 16:28:06.397114 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 16:28:06.397114 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 16:28:06.397114 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 16:28:06.397114 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application with Lock( Jul 01 16:28:06.397114 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 16:28:06.397114 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._enter() Jul 01 16:28:06.397114 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 16:28:06.397114 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 16:28:06.397963 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 16:28:06.397963 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.creator() Jul 01 16:28:06.397963 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 16:28:06.397963 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application created_value = creator( Jul 01 16:28:06.397963 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 01 16:28:06.397963 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 01 16:28:06.397963 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 01 16:28:06.397963 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 01 16:28:06.397963 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 01 16:28:06.397963 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 01 16:28:06.397963 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 01 16:28:06.397963 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Jul 01 16:28:06.397963 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 407/835] 10.222.0.57 () {56 vars in 1102 bytes} [Fri Jul 1 16:28:06 2022] GET /identity/v3/roles/service => generated 85 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 16:28:06.403574 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5486f8-4c28-4dd2-94ec-a7bd4e6ce268 None None] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:28:06.514154 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:28:06.518163 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5486f8-4c28-4dd2-94ec-a7bd4e6ce268 None None] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:28:06.560679 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab3f86b-8863-49ff-8c9d-6d571d99d5e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:28:06.561691 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab3f86b-8863-49ff-8c9d-6d571d99d5e6 None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:28:06.562088 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab3f86b-8863-49ff-8c9d-6d571d99d5e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:28:06.562392 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab3f86b-8863-49ff-8c9d-6d571d99d5e6 None admin] PATH_INFO: `/v3/roles` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:28:06.563173 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab3f86b-8863-49ff-8c9d-6d571d99d5e6 None admin] RBAC: Adding query filter params (name=service) {{(pid=80312) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:28:06.563485 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab3f86b-8863-49ff-8c9d-6d571d99d5e6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:28:06.564943 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab3f86b-8863-49ff-8c9d-6d571d99d5e6 None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:28:06.574485 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 429/836] 10.222.0.57 () {56 vars in 1103 bytes} [Fri Jul 1 16:28:06 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:28:06.583749 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ebc058-c7c7-491c-a438-23b483066ef6 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:28:06.683921 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:28:06.694823 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ebc058-c7c7-491c-a438-23b483066ef6 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:28:06.735132 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef6fa0b-971d-4790-a13e-4ad31a954e59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:28:06.735923 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef6fa0b-971d-4790-a13e-4ad31a954e59 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:28:06.736199 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef6fa0b-971d-4790-a13e-4ad31a954e59 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:28:06.736468 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef6fa0b-971d-4790-a13e-4ad31a954e59 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:28:06.747662 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef6fa0b-971d-4790-a13e-4ad31a954e59 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:28:06.749115 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef6fa0b-971d-4790-a13e-4ad31a954e59 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:28:06.768167 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application [None req-eef6fa0b-971d-4790-a13e-4ad31a954e59 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 16:28:06.768167 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 16:28:06.768167 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 16:28:06.768167 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 16:28:06.768167 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 16:28:06.768167 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 16:28:06.768167 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 16:28:06.768167 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 16:28:06.768167 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 16:28:06.768167 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 16:28:06.768167 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 16:28:06.768167 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 16:28:06.768167 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 01 16:28:06.768167 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 01 16:28:06.768167 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 01 16:28:06.769101 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 01 16:28:06.769101 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 16:28:06.769101 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 16:28:06.769101 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 16:28:06.769101 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 16:28:06.769101 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 16:28:06.769101 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 16:28:06.769101 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 16:28:06.769101 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application with Lock( Jul 01 16:28:06.769101 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 16:28:06.769101 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._enter() Jul 01 16:28:06.769101 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 16:28:06.769101 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 16:28:06.769101 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 16:28:06.769101 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.creator() Jul 01 16:28:06.769101 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 16:28:06.769101 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application created_value = creator( Jul 01 16:28:06.769982 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 01 16:28:06.769982 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 01 16:28:06.769982 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 16:28:06.769982 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Jul 01 16:28:06.769982 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 408/837] 10.222.0.57 () {56 vars in 1108 bytes} [Fri Jul 1 16:28:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 16:28:06.775945 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab3f86b-8863-49ff-8c9d-6d571d99d5e6 None admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:28:06.903882 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab3f86b-8863-49ff-8c9d-6d571d99d5e6 None admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:28:06.940788 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d29bef6-e511-4671-9484-8578c6284398 None None] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:28:06.940992 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d29bef6-e511-4671-9484-8578c6284398 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:28:06.941273 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d29bef6-e511-4671-9484-8578c6284398 None None] PATH_INFO: `/` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:28:06.941739 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5786327b-a1d7-4488-80ee-0aa6a4c61919 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:28:06.942696 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5786327b-a1d7-4488-80ee-0aa6a4c61919 None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:28:06.943127 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5786327b-a1d7-4488-80ee-0aa6a4c61919 None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:28:06.943685 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 409/838] 10.222.0.57 () {54 vars in 926 bytes} [Fri Jul 1 16:28:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 16:28:06.943857 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5786327b-a1d7-4488-80ee-0aa6a4c61919 None admin] PATH_INFO: `/v3/domains` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:28:06.944773 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5786327b-a1d7-4488-80ee-0aa6a4c61919 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80312) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:28:06.945271 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5786327b-a1d7-4488-80ee-0aa6a4c61919 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:28:06.946828 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5786327b-a1d7-4488-80ee-0aa6a4c61919 None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:28:06.948882 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ada008d-1682-429c-bbcb-eeb8ed236e2b None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:28:06.949174 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ada008d-1682-429c-bbcb-eeb8ed236e2b None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:28:06.949455 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ada008d-1682-429c-bbcb-eeb8ed236e2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:28:06.966463 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 430/839] 10.222.0.57 () {56 vars in 1109 bytes} [Fri Jul 1 16:28:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:28:06.968194 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5786327b-a1d7-4488-80ee-0aa6a4c61919 None admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:28:06.968411 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-6ada008d-1682-429c-bbcb-eeb8ed236e2b None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:28:06.969610 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-6ada008d-1682-429c-bbcb-eeb8ed236e2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:28:06.995682 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=100 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:28:07.066459 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 410/840] 10.222.0.57 () {58 vars in 1023 bytes} [Fri Jul 1 16:28:06 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 01 16:28:07.088718 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ada008d-1682-429c-bbcb-eeb8ed236e2b None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:28:07.112907 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5786327b-a1d7-4488-80ee-0aa6a4c61919 None admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:28:07.160936 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e394857b-c9b0-440d-8cbb-b610ea514480 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:28:07.163487 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e394857b-c9b0-440d-8cbb-b610ea514480 None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:28:07.164043 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e394857b-c9b0-440d-8cbb-b610ea514480 None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:28:07.164547 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e394857b-c9b0-440d-8cbb-b610ea514480 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:28:07.174850 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e394857b-c9b0-440d-8cbb-b610ea514480 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:28:07.178386 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e394857b-c9b0-440d-8cbb-b610ea514480 None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:28:07.192841 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:28:07.198470 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application [None req-e394857b-c9b0-440d-8cbb-b610ea514480 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 01 16:28:07.198470 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 16:28:07.198470 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 16:28:07.198470 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 16:28:07.198470 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 16:28:07.198470 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 16:28:07.198470 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 16:28:07.198470 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 16:28:07.198470 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 16:28:07.198470 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 16:28:07.198470 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 16:28:07.198470 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 16:28:07.198470 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 01 16:28:07.198470 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 01 16:28:07.198470 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 01 16:28:07.198470 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 01 16:28:07.199430 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 16:28:07.199430 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 16:28:07.199430 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 01 16:28:07.199430 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 01 16:28:07.199430 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 01 16:28:07.199430 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 01 16:28:07.199430 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 16:28:07.199430 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 16:28:07.199430 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 16:28:07.199430 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 16:28:07.199430 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 16:28:07.199430 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application with Lock( Jul 01 16:28:07.199430 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 16:28:07.199430 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self._enter() Jul 01 16:28:07.199430 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 16:28:07.199430 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 16:28:07.199430 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 16:28:07.199430 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.creator() Jul 01 16:28:07.200465 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 16:28:07.200465 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application created_value = creator( Jul 01 16:28:07.200465 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 01 16:28:07.200465 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 01 16:28:07.200465 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 01 16:28:07.200465 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 01 16:28:07.200465 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 01 16:28:07.200465 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 01 16:28:07.200465 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 01 16:28:07.200465 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application Jul 01 16:28:07.203388 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 431/841] 10.222.0.57 () {56 vars in 1117 bytes} [Fri Jul 1 16:28:06 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 01 16:28:07.214402 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e394857b-c9b0-440d-8cbb-b610ea514480 None admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:28:07.226095 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ada008d-1682-429c-bbcb-eeb8ed236e2b None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:28:07.274258 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77644c9c-3b0e-4209-a05a-41098a9d0f4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:28:07.275110 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77644c9c-3b0e-4209-a05a-41098a9d0f4c None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:28:07.275397 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77644c9c-3b0e-4209-a05a-41098a9d0f4c None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:28:07.275675 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77644c9c-3b0e-4209-a05a-41098a9d0f4c None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:28:07.276342 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77644c9c-3b0e-4209-a05a-41098a9d0f4c None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:28:07.277777 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77644c9c-3b0e-4209-a05a-41098a9d0f4c None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:28:07.292505 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application [None req-77644c9c-3b0e-4209-a05a-41098a9d0f4c None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 01 16:28:07.292505 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 16:28:07.292505 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 01 16:28:07.292505 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 01 16:28:07.292505 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 01 16:28:07.292505 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 01 16:28:07.292505 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 01 16:28:07.292505 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 01 16:28:07.292505 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 01 16:28:07.292505 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Jul 01 16:28:07.292505 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 01 16:28:07.292505 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Jul 01 16:28:07.292505 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 16:28:07.292505 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 16:28:07.292505 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 16:28:07.292505 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 16:28:07.292505 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 16:28:07.293675 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 16:28:07.293675 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 16:28:07.293675 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 16:28:07.293675 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 16:28:07.293675 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 16:28:07.293675 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 16:28:07.293675 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 01 16:28:07.293675 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 01 16:28:07.293675 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 01 16:28:07.293675 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 01 16:28:07.293675 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 16:28:07.293675 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 16:28:07.293675 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 16:28:07.293675 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 16:28:07.293675 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 16:28:07.293675 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 16:28:07.294564 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 16:28:07.294564 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application with Lock( Jul 01 16:28:07.294564 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 16:28:07.294564 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._enter() Jul 01 16:28:07.294564 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 16:28:07.294564 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 16:28:07.294564 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 16:28:07.294564 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.creator() Jul 01 16:28:07.294564 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 16:28:07.294564 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application created_value = creator( Jul 01 16:28:07.294564 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 01 16:28:07.294564 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 01 16:28:07.294564 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 01 16:28:07.294564 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Jul 01 16:28:07.295384 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 411/842] 10.222.0.57 () {56 vars in 1114 bytes} [Fri Jul 1 16:28:07 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 01 16:28:07.302985 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77644c9c-3b0e-4209-a05a-41098a9d0f4c None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:28:07.316992 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=100 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:28:07.340305 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e394857b-c9b0-440d-8cbb-b610ea514480 None admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:28:07.377140 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6347b28-141d-41fb-9b96-50c271d58e4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:28:07.377848 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6347b28-141d-41fb-9b96-50c271d58e4d None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:28:07.378140 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6347b28-141d-41fb-9b96-50c271d58e4d None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:28:07.378463 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6347b28-141d-41fb-9b96-50c271d58e4d None admin] PATH_INFO: `/v3/users` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:28:07.379360 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6347b28-141d-41fb-9b96-50c271d58e4d None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80312) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:28:07.379714 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6347b28-141d-41fb-9b96-50c271d58e4d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:28:07.381714 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6347b28-141d-41fb-9b96-50c271d58e4d None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:28:07.406157 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 432/843] 10.222.0.57 () {56 vars in 1149 bytes} [Fri Jul 1 16:28:07 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:28:07.407600 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6347b28-141d-41fb-9b96-50c271d58e4d None admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:28:07.439997 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77644c9c-3b0e-4209-a05a-41098a9d0f4c None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:28:07.482530 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8fd42a-4d3a-4219-944b-039cf1469013 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:28:07.483632 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8fd42a-4d3a-4219-944b-039cf1469013 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:28:07.484129 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8fd42a-4d3a-4219-944b-039cf1469013 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:28:07.484676 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8fd42a-4d3a-4219-944b-039cf1469013 None admin] PATH_INFO: `/v3/services` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:28:07.485689 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b8fd42a-4d3a-4219-944b-039cf1469013 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:28:07.486256 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b8fd42a-4d3a-4219-944b-039cf1469013 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:28:07.488071 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b8fd42a-4d3a-4219-944b-039cf1469013 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:28:07.499882 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 412/844] 10.222.0.57 () {56 vars in 1114 bytes} [Fri Jul 1 16:28:07 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:28:07.501458 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4c77bf-7253-4dee-8f90-30c474b3cdc9 None None] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:28:07.501882 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4c77bf-7253-4dee-8f90-30c474b3cdc9 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:28:07.502089 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4c77bf-7253-4dee-8f90-30c474b3cdc9 None None] PATH_INFO: `/` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:28:07.503703 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 413/845] 10.222.0.57 () {54 vars in 926 bytes} [Fri Jul 1 16:28:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 16:28:07.508039 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4c77bf-7253-4dee-8f90-30c474b3cdc9 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:28:07.508039 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:28:07.518904 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6347b28-141d-41fb-9b96-50c271d58e4d None admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:28:07.562930 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485a3a8c-ae45-4d11-8299-284f76f96c17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:28:07.563633 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485a3a8c-ae45-4d11-8299-284f76f96c17 None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:28:07.563950 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485a3a8c-ae45-4d11-8299-284f76f96c17 None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:28:07.564606 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485a3a8c-ae45-4d11-8299-284f76f96c17 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:28:07.582178 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=100 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:28:07.634923 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4c77bf-7253-4dee-8f90-30c474b3cdc9 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:28:07.659393 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-485a3a8c-ae45-4d11-8299-284f76f96c17 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:28:07.661070 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-485a3a8c-ae45-4d11-8299-284f76f96c17 None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:28:07.674568 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e129232-65b5-4f66-8f44-1d4067c9c00a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:28:07.675233 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e129232-65b5-4f66-8f44-1d4067c9c00a None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:28:07.675516 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e129232-65b5-4f66-8f44-1d4067c9c00a None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:28:07.675791 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e129232-65b5-4f66-8f44-1d4067c9c00a None admin] PATH_INFO: `/v3/services` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:28:07.676499 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e129232-65b5-4f66-8f44-1d4067c9c00a None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:28:07.676851 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e129232-65b5-4f66-8f44-1d4067c9c00a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:28:07.678548 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e129232-65b5-4f66-8f44-1d4067c9c00a None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:28:07.692004 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 414/846] 10.222.0.57 () {56 vars in 1114 bytes} [Fri Jul 1 16:28:07 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:28:07.692004 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23123555-a767-4560-b0b6-65a8c58e223d None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:28:07.692004 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23123555-a767-4560-b0b6-65a8c58e223d None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:28:07.692004 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23123555-a767-4560-b0b6-65a8c58e223d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:28:07.707409 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-23123555-a767-4560-b0b6-65a8c58e223d None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:28:07.709439 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-23123555-a767-4560-b0b6-65a8c58e223d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:28:07.804448 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:28:07.808353 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 433/847] 10.222.0.57 () {58 vars in 1282 bytes} [Fri Jul 1 16:28:07 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 401 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 01 16:28:07.810199 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbb55ff-c5e2-4ad7-a9ca-96382df8f329 None None] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:28:07.810546 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbb55ff-c5e2-4ad7-a9ca-96382df8f329 None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:28:07.810848 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbb55ff-c5e2-4ad7-a9ca-96382df8f329 None None] PATH_INFO: `/` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:28:07.813072 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 434/848] 10.222.0.57 () {54 vars in 926 bytes} [Fri Jul 1 16:28:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 16:28:07.813720 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dbb55ff-c5e2-4ad7-a9ca-96382df8f329 None None] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:28:07.819756 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 415/849] 10.222.0.57 () {58 vars in 1023 bytes} [Fri Jul 1 16:28:07 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 01 16:28:07.820977 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7600c4ec-34cb-476d-bf1c-bce0d0666445 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:28:07.821395 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7600c4ec-34cb-476d-bf1c-bce0d0666445 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:28:07.821778 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7600c4ec-34cb-476d-bf1c-bce0d0666445 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:28:07.841518 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-7600c4ec-34cb-476d-bf1c-bce0d0666445 None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:28:07.843284 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-7600c4ec-34cb-476d-bf1c-bce0d0666445 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:28:07.877901 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=100 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:28:07.925332 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 416/850] 10.222.0.57 () {58 vars in 1023 bytes} [Fri Jul 1 16:28:07 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 01 16:28:07.926193 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7600c4ec-34cb-476d-bf1c-bce0d0666445 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:28:07.955288 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dbb55ff-c5e2-4ad7-a9ca-96382df8f329 None None] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:28:07.994569 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54e364d-68b9-47fa-827b-1907e4353354 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:28:07.995370 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54e364d-68b9-47fa-827b-1907e4353354 None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:28:07.995653 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54e364d-68b9-47fa-827b-1907e4353354 None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:28:07.995938 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54e364d-68b9-47fa-827b-1907e4353354 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:28:08.007098 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a54e364d-68b9-47fa-827b-1907e4353354 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:28:08.008550 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a54e364d-68b9-47fa-827b-1907e4353354 None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:28:08.027864 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application [None req-a54e364d-68b9-47fa-827b-1907e4353354 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 16:28:08.027864 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 16:28:08.027864 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 16:28:08.027864 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 16:28:08.027864 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 16:28:08.027864 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 16:28:08.027864 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 16:28:08.027864 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 16:28:08.027864 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 16:28:08.027864 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 16:28:08.027864 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 16:28:08.027864 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 16:28:08.027864 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 01 16:28:08.027864 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 01 16:28:08.027864 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 01 16:28:08.029045 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 01 16:28:08.029045 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 16:28:08.029045 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 16:28:08.029045 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 16:28:08.029045 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 16:28:08.029045 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 16:28:08.029045 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 16:28:08.029045 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 16:28:08.029045 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application with Lock( Jul 01 16:28:08.029045 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 16:28:08.029045 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self._enter() Jul 01 16:28:08.029045 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 16:28:08.029045 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 16:28:08.029045 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 16:28:08.029045 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.creator() Jul 01 16:28:08.029045 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 16:28:08.029045 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application created_value = creator( Jul 01 16:28:08.030204 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 01 16:28:08.030204 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 01 16:28:08.030204 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 16:28:08.030204 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application Jul 01 16:28:08.034692 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 435/851] 10.222.0.57 () {56 vars in 1108 bytes} [Fri Jul 1 16:28:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 16:28:08.035479 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54e364d-68b9-47fa-827b-1907e4353354 None admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:28:08.056384 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7600c4ec-34cb-476d-bf1c-bce0d0666445 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:28:08.087769 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:28:08.099321 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844105c6-5183-4f2a-8110-9bfe2d57a117 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:28:08.100253 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844105c6-5183-4f2a-8110-9bfe2d57a117 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:28:08.100518 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844105c6-5183-4f2a-8110-9bfe2d57a117 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:28:08.100817 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844105c6-5183-4f2a-8110-9bfe2d57a117 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:28:08.101536 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-844105c6-5183-4f2a-8110-9bfe2d57a117 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:28:08.103076 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-844105c6-5183-4f2a-8110-9bfe2d57a117 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:28:08.117541 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application [None req-844105c6-5183-4f2a-8110-9bfe2d57a117 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 01 16:28:08.117541 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 16:28:08.117541 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 01 16:28:08.117541 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 01 16:28:08.117541 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 01 16:28:08.117541 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 01 16:28:08.117541 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 01 16:28:08.117541 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 01 16:28:08.117541 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 01 16:28:08.117541 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Jul 01 16:28:08.117541 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 01 16:28:08.117541 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Jul 01 16:28:08.117541 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 16:28:08.117541 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 16:28:08.117541 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 16:28:08.117541 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 16:28:08.118574 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 16:28:08.118574 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 16:28:08.118574 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 16:28:08.118574 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 16:28:08.118574 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 16:28:08.118574 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 16:28:08.118574 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 16:28:08.118574 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 01 16:28:08.118574 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 01 16:28:08.118574 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 01 16:28:08.118574 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 01 16:28:08.118574 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 16:28:08.118574 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 16:28:08.118574 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 16:28:08.118574 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 16:28:08.118574 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 16:28:08.119458 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 16:28:08.119458 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 16:28:08.119458 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application with Lock( Jul 01 16:28:08.119458 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 16:28:08.119458 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._enter() Jul 01 16:28:08.119458 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 16:28:08.119458 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 16:28:08.119458 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 16:28:08.119458 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.creator() Jul 01 16:28:08.119458 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 16:28:08.119458 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application created_value = creator( Jul 01 16:28:08.119458 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 01 16:28:08.119458 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 01 16:28:08.119458 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 01 16:28:08.119458 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Jul 01 16:28:08.123748 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 417/852] 10.222.0.57 () {56 vars in 1132 bytes} [Fri Jul 1 16:28:07 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 16:28:08.124659 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844105c6-5183-4f2a-8110-9bfe2d57a117 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:28:08.153335 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=100 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:28:08.167725 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54e364d-68b9-47fa-827b-1907e4353354 None admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:28:08.227712 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3199fd5c-09f7-4e66-b78d-455cb5ff7f6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:28:08.228797 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3199fd5c-09f7-4e66-b78d-455cb5ff7f6c None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:28:08.229083 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3199fd5c-09f7-4e66-b78d-455cb5ff7f6c None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:28:08.229367 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3199fd5c-09f7-4e66-b78d-455cb5ff7f6c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:28:08.242632 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3199fd5c-09f7-4e66-b78d-455cb5ff7f6c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 01 16:28:08.248818 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844105c6-5183-4f2a-8110-9bfe2d57a117 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:28:08.250738 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application [None req-3199fd5c-09f7-4e66-b78d-455cb5ff7f6c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 01 16:28:08.250738 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 16:28:08.250738 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 16:28:08.250738 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 16:28:08.250738 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 16:28:08.250738 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 16:28:08.250738 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 16:28:08.250738 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 16:28:08.250738 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 16:28:08.250738 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 16:28:08.250738 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 16:28:08.250738 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 16:28:08.250738 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 01 16:28:08.250738 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 01 16:28:08.250738 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 01 16:28:08.250738 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 01 16:28:08.251959 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 01 16:28:08.251959 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 01 16:28:08.251959 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 01 16:28:08.251959 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 01 16:28:08.251959 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 16:28:08.251959 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 16:28:08.251959 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 16:28:08.251959 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 16:28:08.251959 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 16:28:08.251959 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 16:28:08.251959 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 16:28:08.251959 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application with Lock( Jul 01 16:28:08.251959 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 16:28:08.251959 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self._enter() Jul 01 16:28:08.251959 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 16:28:08.251959 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 16:28:08.255919 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 16:28:08.255919 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.creator() Jul 01 16:28:08.255919 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 16:28:08.255919 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application created_value = creator( Jul 01 16:28:08.255919 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 01 16:28:08.255919 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 01 16:28:08.255919 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 01 16:28:08.255919 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 01 16:28:08.255919 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 01 16:28:08.255919 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 01 16:28:08.255919 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 01 16:28:08.255919 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application Jul 01 16:28:08.255919 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 436/853] 10.222.0.57 () {56 vars in 1096 bytes} [Fri Jul 1 16:28:08 2022] GET /identity/v3/roles/admin => generated 83 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 16:28:08.255919 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3199fd5c-09f7-4e66-b78d-455cb5ff7f6c None admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:28:08.300255 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210e6df3-7b14-4519-b50a-f532e9153a92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:28:08.301342 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210e6df3-7b14-4519-b50a-f532e9153a92 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:28:08.301767 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210e6df3-7b14-4519-b50a-f532e9153a92 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:28:08.302785 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210e6df3-7b14-4519-b50a-f532e9153a92 None admin] PATH_INFO: `/v3/domains` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:28:08.303783 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-210e6df3-7b14-4519-b50a-f532e9153a92 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:28:08.303967 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-210e6df3-7b14-4519-b50a-f532e9153a92 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:28:08.305457 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-210e6df3-7b14-4519-b50a-f532e9153a92 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:28:08.326169 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 418/854] 10.222.0.57 () {56 vars in 1109 bytes} [Fri Jul 1 16:28:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:28:08.326169 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210e6df3-7b14-4519-b50a-f532e9153a92 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:28:08.377655 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3199fd5c-09f7-4e66-b78d-455cb5ff7f6c None admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:28:08.393245 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:28:08.415000 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7101e9-dc8a-4380-989a-220f1c2c2ee1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:28:08.415679 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7101e9-dc8a-4380-989a-220f1c2c2ee1 None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:28:08.415966 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7101e9-dc8a-4380-989a-220f1c2c2ee1 None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:28:08.416973 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7101e9-dc8a-4380-989a-220f1c2c2ee1 None admin] PATH_INFO: `/v3/services` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:28:08.416973 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e7101e9-dc8a-4380-989a-220f1c2c2ee1 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80312) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:28:08.417269 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e7101e9-dc8a-4380-989a-220f1c2c2ee1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:28:08.418683 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e7101e9-dc8a-4380-989a-220f1c2c2ee1 None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:28:08.425064 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 437/855] 10.222.0.57 () {56 vars in 1126 bytes} [Fri Jul 1 16:28:08 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:28:08.425921 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7101e9-dc8a-4380-989a-220f1c2c2ee1 None admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:28:08.449810 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=100 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:28:08.453545 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210e6df3-7b14-4519-b50a-f532e9153a92 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:28:08.493708 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b892b28-168a-4e9c-8c07-e9fe79c0d229 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:28:08.494547 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b892b28-168a-4e9c-8c07-e9fe79c0d229 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:28:08.494950 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b892b28-168a-4e9c-8c07-e9fe79c0d229 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:28:08.495306 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b892b28-168a-4e9c-8c07-e9fe79c0d229 None admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:28:08.496482 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b892b28-168a-4e9c-8c07-e9fe79c0d229 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:28:08.496895 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b892b28-168a-4e9c-8c07-e9fe79c0d229 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:28:08.498482 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b892b28-168a-4e9c-8c07-e9fe79c0d229 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:28:08.510629 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 419/856] 10.222.0.57 () {56 vars in 1099 bytes} [Fri Jul 1 16:28:08 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:28:08.511600 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b892b28-168a-4e9c-8c07-e9fe79c0d229 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:28:08.556604 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7101e9-dc8a-4380-989a-220f1c2c2ee1 None admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:28:08.600339 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f47e071-e793-4da9-850b-648884226db2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:28:08.601526 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f47e071-e793-4da9-850b-648884226db2 None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:28:08.601832 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f47e071-e793-4da9-850b-648884226db2 None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:28:08.602543 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f47e071-e793-4da9-850b-648884226db2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:28:08.611809 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f47e071-e793-4da9-850b-648884226db2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:28:08.614256 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f47e071-e793-4da9-850b-648884226db2 None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:28:08.626365 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b892b28-168a-4e9c-8c07-e9fe79c0d229 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:28:08.637313 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application [None req-2f47e071-e793-4da9-850b-648884226db2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 01 16:28:08.637313 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 16:28:08.637313 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 16:28:08.637313 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 16:28:08.637313 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 16:28:08.637313 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 16:28:08.637313 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 16:28:08.637313 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 16:28:08.637313 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 16:28:08.637313 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 16:28:08.637313 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 16:28:08.637313 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 16:28:08.637313 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 01 16:28:08.637313 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 01 16:28:08.637313 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 01 16:28:08.638437 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 01 16:28:08.638437 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 16:28:08.638437 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 16:28:08.638437 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 16:28:08.638437 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 16:28:08.638437 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 16:28:08.638437 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 16:28:08.638437 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 16:28:08.638437 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application with Lock( Jul 01 16:28:08.638437 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 16:28:08.638437 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self._enter() Jul 01 16:28:08.638437 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 16:28:08.638437 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 16:28:08.638437 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 16:28:08.638437 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.creator() Jul 01 16:28:08.638437 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 16:28:08.638437 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application created_value = creator( Jul 01 16:28:08.639522 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 01 16:28:08.639522 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 01 16:28:08.639522 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 01 16:28:08.639522 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 01 16:28:08.639522 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 01 16:28:08.639522 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 01 16:28:08.639522 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 01 16:28:08.639522 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application Jul 01 16:28:08.642652 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 438/857] 10.222.0.57 () {56 vars in 1111 bytes} [Fri Jul 1 16:28:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 16:28:08.643560 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f47e071-e793-4da9-850b-648884226db2 None admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:28:08.647956 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:28:08.679533 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb87b285-e7ce-497f-9b3e-e1f89aa2cf31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:28:08.680273 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb87b285-e7ce-497f-9b3e-e1f89aa2cf31 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:28:08.680488 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb87b285-e7ce-497f-9b3e-e1f89aa2cf31 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:28:08.680763 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb87b285-e7ce-497f-9b3e-e1f89aa2cf31 None admin] PATH_INFO: `/v3/services` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:28:08.681573 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb87b285-e7ce-497f-9b3e-e1f89aa2cf31 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:28:08.682035 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb87b285-e7ce-497f-9b3e-e1f89aa2cf31 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:28:08.684479 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb87b285-e7ce-497f-9b3e-e1f89aa2cf31 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:28:08.692198 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 420/858] 10.222.0.57 () {56 vars in 1126 bytes} [Fri Jul 1 16:28:08 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:28:08.693873 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb87b285-e7ce-497f-9b3e-e1f89aa2cf31 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:28:08.755867 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=100 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:28:08.792885 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f47e071-e793-4da9-850b-648884226db2 None admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:28:08.851169 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35dc413f-b49d-487a-b01c-3675e10aaf7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:28:08.852334 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35dc413f-b49d-487a-b01c-3675e10aaf7d None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:28:08.852625 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35dc413f-b49d-487a-b01c-3675e10aaf7d None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:28:08.852902 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35dc413f-b49d-487a-b01c-3675e10aaf7d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:28:08.854222 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb87b285-e7ce-497f-9b3e-e1f89aa2cf31 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:28:08.870784 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35dc413f-b49d-487a-b01c-3675e10aaf7d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:28:08.872401 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35dc413f-b49d-487a-b01c-3675e10aaf7d None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:28:08.898044 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application [None req-35dc413f-b49d-487a-b01c-3675e10aaf7d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 16:28:08.898044 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 16:28:08.898044 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 16:28:08.898044 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 16:28:08.898044 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 16:28:08.898044 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 16:28:08.898044 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 16:28:08.898044 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 16:28:08.898044 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 16:28:08.898044 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 16:28:08.898044 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 16:28:08.898044 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 16:28:08.898044 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 01 16:28:08.898044 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 01 16:28:08.898044 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 01 16:28:08.899118 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 01 16:28:08.899118 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 16:28:08.899118 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 16:28:08.899118 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 16:28:08.899118 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 16:28:08.899118 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 16:28:08.899118 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 16:28:08.899118 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 16:28:08.899118 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application with Lock( Jul 01 16:28:08.899118 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 16:28:08.899118 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self._enter() Jul 01 16:28:08.899118 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 16:28:08.899118 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 16:28:08.899118 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 16:28:08.899118 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.creator() Jul 01 16:28:08.899118 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 16:28:08.899118 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application created_value = creator( Jul 01 16:28:08.900180 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 01 16:28:08.900180 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 01 16:28:08.900180 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 16:28:08.900180 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application Jul 01 16:28:08.901658 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 439/859] 10.222.0.57 () {56 vars in 1108 bytes} [Fri Jul 1 16:28:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 16:28:08.907234 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35dc413f-b49d-487a-b01c-3675e10aaf7d None admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:28:08.908998 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42bd803-6f5f-4e79-b3b7-5aca50351b2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:28:08.909765 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42bd803-6f5f-4e79-b3b7-5aca50351b2d None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:28:08.910216 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42bd803-6f5f-4e79-b3b7-5aca50351b2d None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:28:08.910503 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42bd803-6f5f-4e79-b3b7-5aca50351b2d None admin] PATH_INFO: `/v3/projects` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:28:08.911559 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42bd803-6f5f-4e79-b3b7-5aca50351b2d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:28:08.912156 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42bd803-6f5f-4e79-b3b7-5aca50351b2d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:28:08.913673 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42bd803-6f5f-4e79-b3b7-5aca50351b2d None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:28:08.927630 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 421/860] 10.222.0.57 () {56 vars in 1148 bytes} [Fri Jul 1 16:28:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:28:08.933377 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42bd803-6f5f-4e79-b3b7-5aca50351b2d None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:28:08.972702 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:28:09.045045 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35dc413f-b49d-487a-b01c-3675e10aaf7d None admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:28:09.049906 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42bd803-6f5f-4e79-b3b7-5aca50351b2d None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:28:09.085673 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=100 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:28:09.094490 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423a5515-591d-4f83-b797-2939be72424d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:28:09.095256 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423a5515-591d-4f83-b797-2939be72424d None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:28:09.095548 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423a5515-591d-4f83-b797-2939be72424d None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:28:09.095830 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423a5515-591d-4f83-b797-2939be72424d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:28:09.096390 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f351a2-1b1f-4a9a-9ba5-86239297b15e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:28:09.096670 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-423a5515-591d-4f83-b797-2939be72424d None admin] RBAC: Adding query filter params (role.id=e8809b67c9dc471daa1c2bd42c9a2baf, scope.project.id=e0788e6725a84e57a4174284c1845755, user.id=c4de1b1eeaed4fc5ae7200b2c7a90957) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:28:09.096988 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-423a5515-591d-4f83-b797-2939be72424d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:28:09.097693 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f351a2-1b1f-4a9a-9ba5-86239297b15e None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:28:09.098175 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f351a2-1b1f-4a9a-9ba5-86239297b15e None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:28:09.098645 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-423a5515-591d-4f83-b797-2939be72424d None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:28:09.100111 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f351a2-1b1f-4a9a-9ba5-86239297b15e None admin] PATH_INFO: `/v3/domains` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:28:09.101498 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f351a2-1b1f-4a9a-9ba5-86239297b15e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80312) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:28:09.102036 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f351a2-1b1f-4a9a-9ba5-86239297b15e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:28:09.104390 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f351a2-1b1f-4a9a-9ba5-86239297b15e None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:28:09.106199 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 422/861] 10.222.0.57 () {56 vars in 1374 bytes} [Fri Jul 1 16:28:08 2022] GET /identity/v3/role_assignments?user.id=c4de1b1eeaed4fc5ae7200b2c7a90957&scope.project.id=e0788e6725a84e57a4174284c1845755&role.id=e8809b67c9dc471daa1c2bd42c9a2baf => generated 608 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 01 16:28:09.118820 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 440/862] 10.222.0.57 () {56 vars in 1109 bytes} [Fri Jul 1 16:28:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 01 16:28:09.126640 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423a5515-591d-4f83-b797-2939be72424d None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:28:09.251238 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423a5515-591d-4f83-b797-2939be72424d None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:28:09.253962 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:28:09.296304 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ec44ce-03c3-46d5-a711-ec055a3bd4c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:28:09.297387 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ec44ce-03c3-46d5-a711-ec055a3bd4c6 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:28:09.297879 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ec44ce-03c3-46d5-a711-ec055a3bd4c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:28:09.298307 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ec44ce-03c3-46d5-a711-ec055a3bd4c6 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:28:09.308829 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ec44ce-03c3-46d5-a711-ec055a3bd4c6 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:28:09.310622 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ec44ce-03c3-46d5-a711-ec055a3bd4c6 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:28:09.326136 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application [None req-f9ec44ce-03c3-46d5-a711-ec055a3bd4c6 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 01 16:28:09.326136 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 16:28:09.326136 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 16:28:09.326136 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 16:28:09.326136 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 16:28:09.326136 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 16:28:09.326136 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 16:28:09.326136 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 16:28:09.326136 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 16:28:09.326136 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 16:28:09.326136 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 16:28:09.326136 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 16:28:09.326136 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 01 16:28:09.326136 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 01 16:28:09.326136 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 01 16:28:09.326136 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 01 16:28:09.327370 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 16:28:09.327370 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 16:28:09.327370 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 01 16:28:09.327370 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 01 16:28:09.327370 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 01 16:28:09.327370 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 01 16:28:09.327370 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 16:28:09.327370 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 16:28:09.327370 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 16:28:09.327370 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 16:28:09.327370 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 16:28:09.327370 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application with Lock( Jul 01 16:28:09.327370 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 16:28:09.327370 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._enter() Jul 01 16:28:09.327370 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 16:28:09.327370 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 16:28:09.327370 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 16:28:09.327370 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.creator() Jul 01 16:28:09.328904 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 16:28:09.328904 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application created_value = creator( Jul 01 16:28:09.328904 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 01 16:28:09.328904 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 01 16:28:09.328904 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 01 16:28:09.328904 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 01 16:28:09.328904 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 01 16:28:09.328904 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 01 16:28:09.328904 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 01 16:28:09.328904 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Jul 01 16:28:09.328904 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 423/863] 10.222.0.57 () {56 vars in 1096 bytes} [Fri Jul 1 16:28:09 2022] GET /identity/v3/users/swift => generated 83 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 01 16:28:09.335870 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f351a2-1b1f-4a9a-9ba5-86239297b15e None admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:28:09.457935 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f351a2-1b1f-4a9a-9ba5-86239297b15e None admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:28:09.496638 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa44383-7b57-44d7-bf04-5fe4f780bad3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:28:09.497320 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa44383-7b57-44d7-bf04-5fe4f780bad3 None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:28:09.497603 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa44383-7b57-44d7-bf04-5fe4f780bad3 None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:28:09.497881 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa44383-7b57-44d7-bf04-5fe4f780bad3 None admin] PATH_INFO: `/v3/users` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:28:09.498825 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa44383-7b57-44d7-bf04-5fe4f780bad3 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80312) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:28:09.499186 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa44383-7b57-44d7-bf04-5fe4f780bad3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:28:09.500630 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa44383-7b57-44d7-bf04-5fe4f780bad3 None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:28:09.521373 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 441/864] 10.222.0.57 () {56 vars in 1135 bytes} [Fri Jul 1 16:28:09 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:28:09.528855 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ec44ce-03c3-46d5-a711-ec055a3bd4c6 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:28:09.639970 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ec44ce-03c3-46d5-a711-ec055a3bd4c6 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:28:09.676453 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e56d65-14d8-4edc-9c11-8a1506020861 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:28:09.677901 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e56d65-14d8-4edc-9c11-8a1506020861 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:28:09.678148 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e56d65-14d8-4edc-9c11-8a1506020861 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:28:09.678456 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e56d65-14d8-4edc-9c11-8a1506020861 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:28:09.711552 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:28:09.753880 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e56d65-14d8-4edc-9c11-8a1506020861 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:28:09.756222 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e56d65-14d8-4edc-9c11-8a1506020861 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:28:09.857676 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eef1655-04f1-4c15-8e54-c3ecaaa2e6d5 None None] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:28:09.857966 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eef1655-04f1-4c15-8e54-c3ecaaa2e6d5 None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:28:09.858246 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eef1655-04f1-4c15-8e54-c3ecaaa2e6d5 None None] PATH_INFO: `/` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:28:09.859859 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 442/865] 10.222.0.57 () {54 vars in 926 bytes} [Fri Jul 1 16:28:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 01 16:28:09.867103 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb30b2ac-83c2-4ea2-a916-5e50c7f13b9f None None] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:28:09.867514 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb30b2ac-83c2-4ea2-a916-5e50c7f13b9f None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:28:09.867898 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb30b2ac-83c2-4ea2-a916-5e50c7f13b9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:28:09.869522 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 424/866] 10.222.0.57 () {58 vars in 1282 bytes} [Fri Jul 1 16:28:09 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 01 16:28:09.874945 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e56d65-14d8-4edc-9c11-8a1506020861 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:28:09.888503 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.auth.core [None req-fb30b2ac-83c2-4ea2-a916-5e50c7f13b9f None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80312) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:28:09.890011 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-fb30b2ac-83c2-4ea2-a916-5e50c7f13b9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:28:09.939418 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:28:09.952214 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=100 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:28:09.991486 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e56d65-14d8-4edc-9c11-8a1506020861 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:28:09.994606 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 443/867] 10.222.0.57 () {58 vars in 1023 bytes} [Fri Jul 1 16:28:09 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 01 16:28:10.013162 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb30b2ac-83c2-4ea2-a916-5e50c7f13b9f None None] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:28:10.034442 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260d761e-ed04-451c-ac63-e9f9149fc3b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:28:10.035248 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260d761e-ed04-451c-ac63-e9f9149fc3b9 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:28:10.035560 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260d761e-ed04-451c-ac63-e9f9149fc3b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:28:10.035867 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260d761e-ed04-451c-ac63-e9f9149fc3b9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:28:10.053150 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-260d761e-ed04-451c-ac63-e9f9149fc3b9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:28:10.055692 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-260d761e-ed04-451c-ac63-e9f9149fc3b9 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:28:10.080284 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application [None req-260d761e-ed04-451c-ac63-e9f9149fc3b9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 16:28:10.080284 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 16:28:10.080284 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 16:28:10.080284 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 16:28:10.080284 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 16:28:10.080284 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 16:28:10.080284 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 16:28:10.080284 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 16:28:10.080284 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 16:28:10.080284 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 16:28:10.080284 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 16:28:10.080284 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 16:28:10.080284 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 01 16:28:10.080284 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 01 16:28:10.080284 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 01 16:28:10.081980 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 01 16:28:10.081980 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 16:28:10.081980 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 16:28:10.081980 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 16:28:10.081980 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 16:28:10.081980 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 16:28:10.081980 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 16:28:10.081980 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 16:28:10.081980 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application with Lock( Jul 01 16:28:10.081980 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 16:28:10.081980 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._enter() Jul 01 16:28:10.081980 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 16:28:10.081980 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 16:28:10.081980 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 16:28:10.081980 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.creator() Jul 01 16:28:10.081980 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 16:28:10.081980 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application created_value = creator( Jul 01 16:28:10.083338 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 01 16:28:10.083338 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 01 16:28:10.083338 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 16:28:10.083338 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Jul 01 16:28:10.083338 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 425/868] 10.222.0.57 () {56 vars in 1108 bytes} [Fri Jul 1 16:28:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 16:28:10.089618 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260d761e-ed04-451c-ac63-e9f9149fc3b9 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:28:10.148367 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb30b2ac-83c2-4ea2-a916-5e50c7f13b9f None None] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:28:10.187123 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b449a0-aaeb-41e6-a5aa-1d8d9d0ad539 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:28:10.187990 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b449a0-aaeb-41e6-a5aa-1d8d9d0ad539 None admin] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:28:10.188271 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b449a0-aaeb-41e6-a5aa-1d8d9d0ad539 None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:28:10.188604 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b449a0-aaeb-41e6-a5aa-1d8d9d0ad539 None admin] PATH_INFO: `/v3/services` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:28:10.189653 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b449a0-aaeb-41e6-a5aa-1d8d9d0ad539 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:28:10.191175 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b449a0-aaeb-41e6-a5aa-1d8d9d0ad539 None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:28:10.212752 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 444/869] 10.222.0.57 () {60 vars in 1141 bytes} [Fri Jul 1 16:28:10 2022] POST /identity/v3/services => generated 251 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 16:28:10.218726 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260d761e-ed04-451c-ac63-e9f9149fc3b9 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:28:10.236039 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:28:10.271100 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91db142a-e1d9-45db-8182-4406b0d74014 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:28:10.272141 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91db142a-e1d9-45db-8182-4406b0d74014 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:28:10.272425 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91db142a-e1d9-45db-8182-4406b0d74014 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:28:10.272785 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91db142a-e1d9-45db-8182-4406b0d74014 None admin] PATH_INFO: `/v3/domains` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:28:10.273464 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91db142a-e1d9-45db-8182-4406b0d74014 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:28:10.273772 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91db142a-e1d9-45db-8182-4406b0d74014 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:28:10.275280 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91db142a-e1d9-45db-8182-4406b0d74014 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:28:10.290097 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 426/870] 10.222.0.57 () {56 vars in 1109 bytes} [Fri Jul 1 16:28:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:28:10.295807 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b449a0-aaeb-41e6-a5aa-1d8d9d0ad539 None admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:28:10.340530 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:28:10.434736 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b449a0-aaeb-41e6-a5aa-1d8d9d0ad539 None admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:28:10.474298 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea35035-98ff-488c-b75f-77ef602b13a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:28:10.475187 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea35035-98ff-488c-b75f-77ef602b13a3 None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:28:10.475480 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea35035-98ff-488c-b75f-77ef602b13a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:28:10.475744 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea35035-98ff-488c-b75f-77ef602b13a3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:28:10.483493 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea35035-98ff-488c-b75f-77ef602b13a3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:28:10.485066 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea35035-98ff-488c-b75f-77ef602b13a3 None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:28:10.499931 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application [None req-8ea35035-98ff-488c-b75f-77ef602b13a3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 01 16:28:10.499931 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 16:28:10.499931 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 16:28:10.499931 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 16:28:10.499931 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 16:28:10.499931 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 16:28:10.499931 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 16:28:10.499931 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 16:28:10.499931 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 16:28:10.499931 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 16:28:10.499931 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 16:28:10.499931 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 16:28:10.499931 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 01 16:28:10.499931 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 01 16:28:10.499931 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 01 16:28:10.500896 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 01 16:28:10.500896 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 16:28:10.500896 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 16:28:10.500896 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 16:28:10.500896 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 16:28:10.500896 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 16:28:10.500896 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 16:28:10.500896 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 16:28:10.500896 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application with Lock( Jul 01 16:28:10.500896 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 16:28:10.500896 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self._enter() Jul 01 16:28:10.500896 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 16:28:10.500896 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 16:28:10.500896 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 16:28:10.500896 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.creator() Jul 01 16:28:10.500896 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 16:28:10.500896 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application created_value = creator( Jul 01 16:28:10.501781 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 01 16:28:10.501781 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 01 16:28:10.501781 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 01 16:28:10.501781 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 01 16:28:10.501781 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 01 16:28:10.501781 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 01 16:28:10.501781 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 01 16:28:10.501781 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application Jul 01 16:28:10.501781 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 445/871] 10.222.0.57 () {56 vars in 1111 bytes} [Fri Jul 1 16:28:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 16:28:10.508730 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91db142a-e1d9-45db-8182-4406b0d74014 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:28:10.620735 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91db142a-e1d9-45db-8182-4406b0d74014 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:28:10.657165 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96747be7-655b-4856-8694-a0d4ae4553ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:28:10.657899 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96747be7-655b-4856-8694-a0d4ae4553ff None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:28:10.658229 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96747be7-655b-4856-8694-a0d4ae4553ff None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:28:10.658580 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96747be7-655b-4856-8694-a0d4ae4553ff None admin] PATH_INFO: `/v3/projects` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:28:10.659469 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96747be7-655b-4856-8694-a0d4ae4553ff None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:28:10.659811 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96747be7-655b-4856-8694-a0d4ae4553ff None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:28:10.661866 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96747be7-655b-4856-8694-a0d4ae4553ff None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:28:10.675964 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 427/872] 10.222.0.57 () {56 vars in 1148 bytes} [Fri Jul 1 16:28:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:28:10.681988 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea35035-98ff-488c-b75f-77ef602b13a3 None admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:28:10.801107 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=100 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:28:10.808115 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea35035-98ff-488c-b75f-77ef602b13a3 None admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:28:10.845482 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40c8274-1878-477f-a50f-5aeb49a64aa1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:28:10.845482 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40c8274-1878-477f-a50f-5aeb49a64aa1 None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:28:10.845482 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40c8274-1878-477f-a50f-5aeb49a64aa1 None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:28:10.845482 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40c8274-1878-477f-a50f-5aeb49a64aa1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:28:10.846324 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d40c8274-1878-477f-a50f-5aeb49a64aa1 None admin] RBAC: Adding query filter params (role.id=84ee739146934c1ab32c501ff8093644, scope.project.id=e0788e6725a84e57a4174284c1845755, user.id=859fe410c2d2450fb803c6244839f27c) {{(pid=80312) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:28:10.846510 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d40c8274-1878-477f-a50f-5aeb49a64aa1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:28:10.850594 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d40c8274-1878-477f-a50f-5aeb49a64aa1 None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:28:10.855079 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 446/873] 10.222.0.57 () {56 vars in 1374 bytes} [Fri Jul 1 16:28:10 2022] GET /identity/v3/role_assignments?user.id=859fe410c2d2450fb803c6244839f27c&scope.project.id=e0788e6725a84e57a4174284c1845755&role.id=84ee739146934c1ab32c501ff8093644 => generated 608 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:28:10.959313 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc96c66-15e9-4c43-b547-5bd424ad633b None None] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:28:10.959621 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc96c66-15e9-4c43-b547-5bd424ad633b None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:28:10.959794 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc96c66-15e9-4c43-b547-5bd424ad633b None None] PATH_INFO: `/` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:28:10.961504 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 428/874] 10.222.0.57 () {54 vars in 926 bytes} [Fri Jul 1 16:28:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 16:28:10.967541 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbf32ed-cab5-4aa9-b84d-21d81f975b8b None None] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:28:10.967859 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbf32ed-cab5-4aa9-b84d-21d81f975b8b None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:28:10.968106 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbf32ed-cab5-4aa9-b84d-21d81f975b8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:28:10.986916 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.auth.core [None req-1cbf32ed-cab5-4aa9-b84d-21d81f975b8b None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80312) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:28:10.988155 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-1cbf32ed-cab5-4aa9-b84d-21d81f975b8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:28:11.082337 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 447/875] 10.222.0.57 () {58 vars in 1023 bytes} [Fri Jul 1 16:28:10 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 01 16:28:11.104328 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc96c66-15e9-4c43-b547-5bd424ad633b None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:28:11.134723 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:28:11.213389 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc96c66-15e9-4c43-b547-5bd424ad633b None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:28:11.249830 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ac29b7-2d5d-4ef1-8731-28f3f3d220ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:28:11.250585 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ac29b7-2d5d-4ef1-8731-28f3f3d220ba None admin] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:28:11.251194 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ac29b7-2d5d-4ef1-8731-28f3f3d220ba None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:28:11.251471 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ac29b7-2d5d-4ef1-8731-28f3f3d220ba None admin] PATH_INFO: `/v3/services` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:28:11.252210 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ac29b7-2d5d-4ef1-8731-28f3f3d220ba None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:28:11.253570 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ac29b7-2d5d-4ef1-8731-28f3f3d220ba None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:28:11.269198 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 429/876] 10.222.0.57 () {60 vars in 1141 bytes} [Fri Jul 1 16:28:11 2022] POST /identity/v3/services => generated 269 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 16:28:11.539495 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87192672-c53f-48d1-b325-f6a3d90f979a None None] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:28:11.540515 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87192672-c53f-48d1-b325-f6a3d90f979a None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:28:11.541025 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87192672-c53f-48d1-b325-f6a3d90f979a None None] PATH_INFO: `/` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:28:11.542920 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 448/877] 10.222.0.57 () {54 vars in 926 bytes} [Fri Jul 1 16:28:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 16:28:11.548501 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c0162d-0ecf-4b13-bb6b-8a408ea4111a None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:28:11.548950 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c0162d-0ecf-4b13-bb6b-8a408ea4111a None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:28:11.549037 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c0162d-0ecf-4b13-bb6b-8a408ea4111a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:28:11.569945 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-20c0162d-0ecf-4b13-bb6b-8a408ea4111a None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:28:11.571572 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-20c0162d-0ecf-4b13-bb6b-8a408ea4111a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:28:11.651857 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 430/878] 10.222.0.57 () {58 vars in 1023 bytes} [Fri Jul 1 16:28:11 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 01 16:28:11.675961 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87192672-c53f-48d1-b325-f6a3d90f979a None None] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:28:11.775048 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=100 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:28:11.787026 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87192672-c53f-48d1-b325-f6a3d90f979a None None] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:28:11.822054 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb46a40-2389-48f4-abb4-8c18b1a8c2af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:28:11.822822 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb46a40-2389-48f4-abb4-8c18b1a8c2af None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:28:11.823102 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb46a40-2389-48f4-abb4-8c18b1a8c2af None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:28:11.823378 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb46a40-2389-48f4-abb4-8c18b1a8c2af None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:28:11.834481 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb46a40-2389-48f4-abb4-8c18b1a8c2af None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 01 16:28:11.842786 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application [None req-efb46a40-2389-48f4-abb4-8c18b1a8c2af None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 01 16:28:11.842786 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 16:28:11.842786 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 16:28:11.842786 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 16:28:11.842786 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 16:28:11.842786 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 16:28:11.842786 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 16:28:11.842786 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 16:28:11.842786 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 16:28:11.842786 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 16:28:11.842786 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 16:28:11.842786 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 16:28:11.842786 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 01 16:28:11.842786 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 01 16:28:11.842786 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 01 16:28:11.842786 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 01 16:28:11.843813 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 01 16:28:11.843813 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 01 16:28:11.843813 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 01 16:28:11.843813 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 01 16:28:11.843813 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 16:28:11.843813 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 16:28:11.843813 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 16:28:11.843813 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 16:28:11.843813 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 16:28:11.843813 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 16:28:11.843813 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 16:28:11.843813 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application with Lock( Jul 01 16:28:11.843813 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 16:28:11.843813 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self._enter() Jul 01 16:28:11.843813 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 16:28:11.843813 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 16:28:11.844683 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 16:28:11.844683 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.creator() Jul 01 16:28:11.844683 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 16:28:11.844683 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application created_value = creator( Jul 01 16:28:11.844683 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 01 16:28:11.844683 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 01 16:28:11.844683 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 01 16:28:11.844683 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 01 16:28:11.844683 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 01 16:28:11.844683 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 01 16:28:11.844683 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 01 16:28:11.844683 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application Jul 01 16:28:11.847177 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 449/879] 10.222.0.57 () {56 vars in 1120 bytes} [Fri Jul 1 16:28:11 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 16:28:11.853945 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c0162d-0ecf-4b13-bb6b-8a408ea4111a None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:28:11.869791 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:28:11.963752 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c0162d-0ecf-4b13-bb6b-8a408ea4111a None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:28:12.002939 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75fbe084-7f77-4f34-bf3c-1bf4cb9214ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:28:12.004021 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fbe084-7f77-4f34-bf3c-1bf4cb9214ab None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:28:12.004518 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fbe084-7f77-4f34-bf3c-1bf4cb9214ab None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:28:12.005037 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fbe084-7f77-4f34-bf3c-1bf4cb9214ab None admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:28:12.005993 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75fbe084-7f77-4f34-bf3c-1bf4cb9214ab None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:28:12.006539 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75fbe084-7f77-4f34-bf3c-1bf4cb9214ab None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:28:12.008130 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75fbe084-7f77-4f34-bf3c-1bf4cb9214ab None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:28:12.017474 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 431/880] 10.222.0.57 () {56 vars in 1115 bytes} [Fri Jul 1 16:28:11 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:28:12.024153 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb46a40-2389-48f4-abb4-8c18b1a8c2af None admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:28:12.136865 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb46a40-2389-48f4-abb4-8c18b1a8c2af None admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:28:12.173022 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ed27c6-e3f1-4986-9051-457202f87277 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:28:12.174234 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ed27c6-e3f1-4986-9051-457202f87277 None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:28:12.174754 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ed27c6-e3f1-4986-9051-457202f87277 None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:28:12.175234 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ed27c6-e3f1-4986-9051-457202f87277 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:28:12.188031 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ed27c6-e3f1-4986-9051-457202f87277 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:28:12.189828 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ed27c6-e3f1-4986-9051-457202f87277 None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:28:12.209178 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application [None req-98ed27c6-e3f1-4986-9051-457202f87277 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 16:28:12.209178 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 16:28:12.209178 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 16:28:12.209178 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 16:28:12.209178 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 16:28:12.209178 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 16:28:12.209178 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 16:28:12.209178 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 16:28:12.209178 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 16:28:12.209178 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 16:28:12.209178 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 16:28:12.209178 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 16:28:12.209178 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 01 16:28:12.209178 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 01 16:28:12.209178 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 01 16:28:12.210179 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 01 16:28:12.210179 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 16:28:12.210179 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 16:28:12.210179 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 16:28:12.210179 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 16:28:12.210179 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 16:28:12.210179 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 16:28:12.210179 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 16:28:12.210179 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application with Lock( Jul 01 16:28:12.210179 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 16:28:12.210179 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self._enter() Jul 01 16:28:12.210179 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 16:28:12.210179 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 16:28:12.210179 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 16:28:12.210179 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.creator() Jul 01 16:28:12.210179 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 16:28:12.210179 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application created_value = creator( Jul 01 16:28:12.211113 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 01 16:28:12.211113 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 01 16:28:12.211113 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 16:28:12.211113 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application Jul 01 16:28:12.213552 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 450/881] 10.222.0.57 () {56 vars in 1108 bytes} [Fri Jul 1 16:28:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 16:28:12.219282 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75fbe084-7f77-4f34-bf3c-1bf4cb9214ab None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:28:12.313768 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:28:12.342872 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75fbe084-7f77-4f34-bf3c-1bf4cb9214ab None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:28:12.384556 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceca6582-b431-4cd4-96a3-95bb1c98e8be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:28:12.385659 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceca6582-b431-4cd4-96a3-95bb1c98e8be None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:28:12.386220 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceca6582-b431-4cd4-96a3-95bb1c98e8be None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:28:12.386731 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceca6582-b431-4cd4-96a3-95bb1c98e8be None admin] PATH_INFO: `/v3/domains` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:28:12.387726 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceca6582-b431-4cd4-96a3-95bb1c98e8be None admin] RBAC: Adding query filter params (name=Default) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:28:12.388244 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceca6582-b431-4cd4-96a3-95bb1c98e8be None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:28:12.389831 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceca6582-b431-4cd4-96a3-95bb1c98e8be None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:28:12.404388 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 432/882] 10.222.0.57 () {56 vars in 1109 bytes} [Fri Jul 1 16:28:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:28:12.410145 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ed27c6-e3f1-4986-9051-457202f87277 None admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:28:12.414088 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=100 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:28:12.468812 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e98f80-9fbd-4b17-97fe-7d13cefc5c5d None None] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:28:12.469113 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e98f80-9fbd-4b17-97fe-7d13cefc5c5d None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:28:12.469375 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e98f80-9fbd-4b17-97fe-7d13cefc5c5d None None] PATH_INFO: `/` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:28:12.472011 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 433/883] 10.222.0.57 () {54 vars in 926 bytes} [Fri Jul 1 16:28:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 16:28:12.476728 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ea2621-5ad3-4c59-846c-256ad2332bca None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:28:12.477015 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ea2621-5ad3-4c59-846c-256ad2332bca None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:28:12.477301 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ea2621-5ad3-4c59-846c-256ad2332bca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:28:12.503384 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-a0ea2621-5ad3-4c59-846c-256ad2332bca None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:28:12.506913 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-a0ea2621-5ad3-4c59-846c-256ad2332bca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:28:12.537513 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ed27c6-e3f1-4986-9051-457202f87277 None admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:28:12.573510 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982b5ee2-f4c5-4629-8304-39308d963c8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:28:12.574327 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982b5ee2-f4c5-4629-8304-39308d963c8e None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:28:12.574640 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982b5ee2-f4c5-4629-8304-39308d963c8e None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:28:12.574926 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982b5ee2-f4c5-4629-8304-39308d963c8e None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:28:12.586447 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-982b5ee2-f4c5-4629-8304-39308d963c8e None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:28:12.587905 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-982b5ee2-f4c5-4629-8304-39308d963c8e None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:28:12.601069 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 434/884] 10.222.0.57 () {58 vars in 1023 bytes} [Fri Jul 1 16:28:12 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 01 16:28:12.612737 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application [None req-982b5ee2-f4c5-4629-8304-39308d963c8e None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 01 16:28:12.612737 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 16:28:12.612737 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 16:28:12.612737 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 16:28:12.612737 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 16:28:12.612737 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 16:28:12.612737 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 16:28:12.612737 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 16:28:12.612737 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 16:28:12.612737 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 16:28:12.612737 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 16:28:12.612737 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 16:28:12.612737 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 01 16:28:12.612737 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 01 16:28:12.612737 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 01 16:28:12.612737 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 01 16:28:12.613711 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 16:28:12.613711 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 16:28:12.613711 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 01 16:28:12.613711 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 01 16:28:12.613711 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 01 16:28:12.613711 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 01 16:28:12.613711 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 16:28:12.613711 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 16:28:12.613711 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 16:28:12.613711 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 16:28:12.613711 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 16:28:12.613711 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application with Lock( Jul 01 16:28:12.613711 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 16:28:12.613711 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self._enter() Jul 01 16:28:12.613711 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 16:28:12.613711 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 16:28:12.613711 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 16:28:12.613711 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.creator() Jul 01 16:28:12.614734 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 16:28:12.614734 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application created_value = creator( Jul 01 16:28:12.614734 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 01 16:28:12.614734 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 01 16:28:12.614734 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 01 16:28:12.614734 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 01 16:28:12.614734 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 01 16:28:12.614734 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 01 16:28:12.614734 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 01 16:28:12.614734 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application Jul 01 16:28:12.617039 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 451/885] 10.222.0.57 () {56 vars in 1117 bytes} [Fri Jul 1 16:28:12 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 16:28:12.624720 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ea2621-5ad3-4c59-846c-256ad2332bca None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:28:12.637099 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982b5ee2-f4c5-4629-8304-39308d963c8e None admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:28:12.719555 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:28:12.719741 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=100 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:28:12.765492 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ea2621-5ad3-4c59-846c-256ad2332bca None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:28:12.790210 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982b5ee2-f4c5-4629-8304-39308d963c8e None admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:28:12.813755 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0dc868-5f23-422f-ad38-65c19c0f4e76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:28:12.814577 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0dc868-5f23-422f-ad38-65c19c0f4e76 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:28:12.814868 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0dc868-5f23-422f-ad38-65c19c0f4e76 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:28:12.815173 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0dc868-5f23-422f-ad38-65c19c0f4e76 None admin] PATH_INFO: `/v3/users` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:28:12.816341 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a0dc868-5f23-422f-ad38-65c19c0f4e76 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:28:12.816657 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a0dc868-5f23-422f-ad38-65c19c0f4e76 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:28:12.819171 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a0dc868-5f23-422f-ad38-65c19c0f4e76 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:28:12.838335 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85bfa172-394f-405b-b2cc-5e96e8ca8f81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:28:12.840085 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bfa172-394f-405b-b2cc-5e96e8ca8f81 None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:28:12.840887 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bfa172-394f-405b-b2cc-5e96e8ca8f81 None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:28:12.841489 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bfa172-394f-405b-b2cc-5e96e8ca8f81 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:28:12.843613 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85bfa172-394f-405b-b2cc-5e96e8ca8f81 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:28:12.846784 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 435/886] 10.222.0.57 () {56 vars in 1149 bytes} [Fri Jul 1 16:28:12 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:28:12.847773 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0dc868-5f23-422f-ad38-65c19c0f4e76 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:28:12.852483 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85bfa172-394f-405b-b2cc-5e96e8ca8f81 None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:28:12.870714 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application [None req-85bfa172-394f-405b-b2cc-5e96e8ca8f81 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 01 16:28:12.870714 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 16:28:12.870714 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 01 16:28:12.870714 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 01 16:28:12.870714 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 01 16:28:12.870714 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 01 16:28:12.870714 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 01 16:28:12.870714 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 01 16:28:12.870714 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 01 16:28:12.870714 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application Jul 01 16:28:12.870714 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 01 16:28:12.870714 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application Jul 01 16:28:12.870714 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 16:28:12.870714 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 16:28:12.870714 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 16:28:12.870714 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 16:28:12.870714 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 16:28:12.871814 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 16:28:12.871814 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 16:28:12.871814 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 16:28:12.871814 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 16:28:12.871814 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 16:28:12.871814 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 16:28:12.871814 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 01 16:28:12.871814 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 01 16:28:12.871814 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 01 16:28:12.871814 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 01 16:28:12.871814 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 16:28:12.871814 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 16:28:12.871814 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 16:28:12.871814 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 16:28:12.871814 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 16:28:12.871814 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 16:28:12.872654 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 16:28:12.872654 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application with Lock( Jul 01 16:28:12.872654 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 16:28:12.872654 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self._enter() Jul 01 16:28:12.872654 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 16:28:12.872654 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 16:28:12.872654 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 16:28:12.872654 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.creator() Jul 01 16:28:12.872654 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 16:28:12.872654 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application created_value = creator( Jul 01 16:28:12.872654 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 01 16:28:12.872654 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 01 16:28:12.872654 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 01 16:28:12.872654 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application Jul 01 16:28:12.880064 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 452/887] 10.222.0.57 () {56 vars in 1114 bytes} [Fri Jul 1 16:28:12 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 16:28:12.882588 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85bfa172-394f-405b-b2cc-5e96e8ca8f81 None admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:28:12.983070 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0dc868-5f23-422f-ad38-65c19c0f4e76 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:28:13.002685 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85bfa172-394f-405b-b2cc-5e96e8ca8f81 None admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:28:13.015789 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:28:13.029592 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753e2736-0d51-4c37-ae3b-c8502bc5a232 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:28:13.030659 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753e2736-0d51-4c37-ae3b-c8502bc5a232 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:28:13.031175 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753e2736-0d51-4c37-ae3b-c8502bc5a232 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:28:13.031647 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753e2736-0d51-4c37-ae3b-c8502bc5a232 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:28:13.039009 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=100 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:28:13.051279 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b256827b-c22f-4cd6-a481-41b9ee965352 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:28:13.052861 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b256827b-c22f-4cd6-a481-41b9ee965352 None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:28:13.053149 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b256827b-c22f-4cd6-a481-41b9ee965352 None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:28:13.053433 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b256827b-c22f-4cd6-a481-41b9ee965352 None admin] PATH_INFO: `/v3/services` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:28:13.054168 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b256827b-c22f-4cd6-a481-41b9ee965352 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80312) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:28:13.054511 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b256827b-c22f-4cd6-a481-41b9ee965352 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:28:13.056389 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b256827b-c22f-4cd6-a481-41b9ee965352 None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:28:13.067637 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 453/888] 10.222.0.57 () {56 vars in 1114 bytes} [Fri Jul 1 16:28:12 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:28:13.070050 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b256827b-c22f-4cd6-a481-41b9ee965352 None admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:28:13.117368 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-753e2736-0d51-4c37-ae3b-c8502bc5a232 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:28:13.119211 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-753e2736-0d51-4c37-ae3b-c8502bc5a232 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:28:13.207514 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b256827b-c22f-4cd6-a481-41b9ee965352 None admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:28:13.228828 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 436/889] 10.222.0.57 () {58 vars in 1282 bytes} [Fri Jul 1 16:28:12 2022] GET /identity/v3/auth/tokens => generated 1362 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 01 16:28:13.230070 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170d9074-8d9e-4fd2-bf3a-8f0ece370138 None None] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:28:13.230387 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170d9074-8d9e-4fd2-bf3a-8f0ece370138 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:28:13.230670 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170d9074-8d9e-4fd2-bf3a-8f0ece370138 None None] PATH_INFO: `/` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:28:13.236061 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 437/890] 10.222.0.57 () {54 vars in 926 bytes} [Fri Jul 1 16:28:13 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 16:28:13.236805 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170d9074-8d9e-4fd2-bf3a-8f0ece370138 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:28:13.251253 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3a2cfd-fdd3-4beb-95d4-34ff3b689f8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:28:13.252548 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3a2cfd-fdd3-4beb-95d4-34ff3b689f8a None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:28:13.253515 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3a2cfd-fdd3-4beb-95d4-34ff3b689f8a None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:28:13.254011 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3a2cfd-fdd3-4beb-95d4-34ff3b689f8a None admin] PATH_INFO: `/v3/services` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:28:13.256210 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a3a2cfd-fdd3-4beb-95d4-34ff3b689f8a None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80312) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:28:13.256955 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a3a2cfd-fdd3-4beb-95d4-34ff3b689f8a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:28:13.258985 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a3a2cfd-fdd3-4beb-95d4-34ff3b689f8a None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:28:13.264996 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:28:13.268004 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 454/891] 10.222.0.57 () {56 vars in 1114 bytes} [Fri Jul 1 16:28:13 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:28:13.269451 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13dfaa6-2edb-40ce-8126-756edf008da2 None None] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:28:13.269809 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13dfaa6-2edb-40ce-8126-756edf008da2 None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:28:13.270164 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13dfaa6-2edb-40ce-8126-756edf008da2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:28:13.288186 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.auth.core [None req-c13dfaa6-2edb-40ce-8126-756edf008da2 None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80312) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:28:13.289389 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-c13dfaa6-2edb-40ce-8126-756edf008da2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:28:13.341570 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=100 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:28:13.361393 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 455/892] 10.222.0.57 () {58 vars in 1023 bytes} [Fri Jul 1 16:28:13 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 01 16:28:13.362031 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13dfaa6-2edb-40ce-8126-756edf008da2 None None] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:28:13.371917 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170d9074-8d9e-4fd2-bf3a-8f0ece370138 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:28:13.425460 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4a9f41-3143-489b-9c6b-ed0cf7b134e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:28:13.426577 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4a9f41-3143-489b-9c6b-ed0cf7b134e3 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:28:13.427279 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4a9f41-3143-489b-9c6b-ed0cf7b134e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:28:13.427762 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4a9f41-3143-489b-9c6b-ed0cf7b134e3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:28:13.440918 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4a9f41-3143-489b-9c6b-ed0cf7b134e3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:28:13.443213 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4a9f41-3143-489b-9c6b-ed0cf7b134e3 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:28:13.463637 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application [None req-6a4a9f41-3143-489b-9c6b-ed0cf7b134e3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 16:28:13.463637 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 16:28:13.463637 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 16:28:13.463637 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 16:28:13.463637 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 16:28:13.463637 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 16:28:13.463637 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 16:28:13.463637 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 16:28:13.463637 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 16:28:13.463637 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 16:28:13.463637 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 16:28:13.463637 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 16:28:13.463637 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 01 16:28:13.463637 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 01 16:28:13.463637 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 01 16:28:13.464625 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 01 16:28:13.464625 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 16:28:13.464625 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 16:28:13.464625 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 16:28:13.464625 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 16:28:13.464625 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 16:28:13.464625 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 16:28:13.464625 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 16:28:13.464625 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application with Lock( Jul 01 16:28:13.464625 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 16:28:13.464625 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._enter() Jul 01 16:28:13.464625 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 16:28:13.464625 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 16:28:13.464625 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 16:28:13.464625 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.creator() Jul 01 16:28:13.464625 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 16:28:13.464625 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application created_value = creator( Jul 01 16:28:13.465517 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 01 16:28:13.465517 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 01 16:28:13.465517 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 16:28:13.465517 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Jul 01 16:28:13.465517 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 438/893] 10.222.0.57 () {56 vars in 1108 bytes} [Fri Jul 1 16:28:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 16:28:13.465797 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4a9f41-3143-489b-9c6b-ed0cf7b134e3 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:28:13.509446 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13dfaa6-2edb-40ce-8126-756edf008da2 None None] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:28:13.545713 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184540ad-19e3-4b20-ba70-ee562a8f3f86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:28:13.546537 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184540ad-19e3-4b20-ba70-ee562a8f3f86 None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:28:13.546825 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184540ad-19e3-4b20-ba70-ee562a8f3f86 None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:28:13.547098 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184540ad-19e3-4b20-ba70-ee562a8f3f86 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:28:13.547917 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-184540ad-19e3-4b20-ba70-ee562a8f3f86 None admin] RBAC: Adding query filter params (interface=public, service_id=12d1d240ae2743cfbb495d16dcc7467d, region_id=RegionOne) {{(pid=80312) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:28:13.548228 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-184540ad-19e3-4b20-ba70-ee562a8f3f86 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:28:13.549631 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-184540ad-19e3-4b20-ba70-ee562a8f3f86 None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:28:13.551611 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:28:13.555670 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 456/894] 10.222.0.57 () {56 vars in 1251 bytes} [Fri Jul 1 16:28:13 2022] GET /identity/v3/endpoints?service_id=12d1d240ae2743cfbb495d16dcc7467d&interface=public®ion_id=RegionOne => generated 194 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:28:13.556492 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184540ad-19e3-4b20-ba70-ee562a8f3f86 None admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:28:13.580816 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4a9f41-3143-489b-9c6b-ed0cf7b134e3 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:28:13.606456 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=100 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:28:13.621758 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625f6005-575d-496b-9416-350198c5d6ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:28:13.622664 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625f6005-575d-496b-9416-350198c5d6ef None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:28:13.622953 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625f6005-575d-496b-9416-350198c5d6ef None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:28:13.623283 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625f6005-575d-496b-9416-350198c5d6ef None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:28:13.624005 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-625f6005-575d-496b-9416-350198c5d6ef None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:28:13.625450 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-625f6005-575d-496b-9416-350198c5d6ef None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:28:13.637999 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application [None req-625f6005-575d-496b-9416-350198c5d6ef None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 01 16:28:13.637999 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 16:28:13.637999 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 01 16:28:13.637999 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 01 16:28:13.637999 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 01 16:28:13.637999 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 01 16:28:13.637999 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 01 16:28:13.637999 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 01 16:28:13.637999 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 01 16:28:13.637999 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Jul 01 16:28:13.637999 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 01 16:28:13.637999 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Jul 01 16:28:13.637999 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 16:28:13.637999 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 16:28:13.637999 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 16:28:13.637999 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 16:28:13.637999 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 16:28:13.639198 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 16:28:13.639198 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 16:28:13.639198 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 16:28:13.639198 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 16:28:13.639198 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 16:28:13.639198 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 16:28:13.639198 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 01 16:28:13.639198 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 01 16:28:13.639198 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 01 16:28:13.639198 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 01 16:28:13.639198 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 16:28:13.639198 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 16:28:13.639198 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 16:28:13.639198 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 16:28:13.639198 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 16:28:13.639198 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 16:28:13.640057 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 16:28:13.640057 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application with Lock( Jul 01 16:28:13.640057 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 16:28:13.640057 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._enter() Jul 01 16:28:13.640057 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 16:28:13.640057 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 16:28:13.640057 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 16:28:13.640057 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.creator() Jul 01 16:28:13.640057 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 16:28:13.640057 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application created_value = creator( Jul 01 16:28:13.640057 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 01 16:28:13.640057 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 01 16:28:13.640057 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 01 16:28:13.640057 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Jul 01 16:28:13.640057 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 439/895] 10.222.0.57 () {56 vars in 1126 bytes} [Fri Jul 1 16:28:13 2022] GET /identity/v3/services/object-store => generated 93 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 16:28:13.640858 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3efda12-4588-4b4f-993b-89fd03c58579 None None] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:28:13.640858 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3efda12-4588-4b4f-993b-89fd03c58579 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:28:13.640996 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3efda12-4588-4b4f-993b-89fd03c58579 None None] PATH_INFO: `/` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:28:13.642253 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 440/896] 10.222.0.57 () {54 vars in 926 bytes} [Fri Jul 1 16:28:13 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 16:28:13.643004 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3efda12-4588-4b4f-993b-89fd03c58579 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:28:13.664538 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184540ad-19e3-4b20-ba70-ee562a8f3f86 None admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:28:13.700574 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b61b13-3675-4a7a-aef4-9bcb18cd96e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:28:13.701279 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b61b13-3675-4a7a-aef4-9bcb18cd96e8 None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:28:13.701563 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b61b13-3675-4a7a-aef4-9bcb18cd96e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:28:13.701838 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b61b13-3675-4a7a-aef4-9bcb18cd96e8 None admin] PATH_INFO: `/v3/domains` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:28:13.702600 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b61b13-3675-4a7a-aef4-9bcb18cd96e8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80312) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:28:13.703174 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b61b13-3675-4a7a-aef4-9bcb18cd96e8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:28:13.704525 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b61b13-3675-4a7a-aef4-9bcb18cd96e8 None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:28:13.717120 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 457/897] 10.222.0.57 () {56 vars in 1109 bytes} [Fri Jul 1 16:28:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:28:13.718273 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b61b13-3675-4a7a-aef4-9bcb18cd96e8 None admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:28:13.760120 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3efda12-4588-4b4f-993b-89fd03c58579 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:28:13.795413 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015cb74c-05c3-44a1-9b0d-bb5f6da1fe6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:28:13.796078 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015cb74c-05c3-44a1-9b0d-bb5f6da1fe6f None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:28:13.796367 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015cb74c-05c3-44a1-9b0d-bb5f6da1fe6f None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:28:13.796650 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015cb74c-05c3-44a1-9b0d-bb5f6da1fe6f None admin] PATH_INFO: `/v3/services` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:28:13.797340 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-015cb74c-05c3-44a1-9b0d-bb5f6da1fe6f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:28:13.798744 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-015cb74c-05c3-44a1-9b0d-bb5f6da1fe6f None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:28:13.805008 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 441/898] 10.222.0.57 () {56 vars in 1087 bytes} [Fri Jul 1 16:28:13 2022] GET /identity/v3/services => generated 1268 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 16:28:13.806313 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e11ae1-4900-43ed-a824-7855afa3b230 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:28:13.806634 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e11ae1-4900-43ed-a824-7855afa3b230 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:28:13.806927 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e11ae1-4900-43ed-a824-7855afa3b230 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:28:13.820438 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:28:13.826189 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b61b13-3675-4a7a-aef4-9bcb18cd96e8 None admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:28:13.840783 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=100 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:28:13.847998 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-18e11ae1-4900-43ed-a824-7855afa3b230 None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:28:13.850440 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-18e11ae1-4900-43ed-a824-7855afa3b230 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:28:13.867951 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602bdb14-19ed-4124-8c67-90a2c7a96615 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:28:13.868793 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602bdb14-19ed-4124-8c67-90a2c7a96615 None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:28:13.869148 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602bdb14-19ed-4124-8c67-90a2c7a96615 None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:28:13.869507 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602bdb14-19ed-4124-8c67-90a2c7a96615 None admin] PATH_INFO: `/v3/services` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:28:13.870322 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-602bdb14-19ed-4124-8c67-90a2c7a96615 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80312) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:28:13.870813 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-602bdb14-19ed-4124-8c67-90a2c7a96615 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:28:13.872296 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-602bdb14-19ed-4124-8c67-90a2c7a96615 None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:28:13.882654 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 458/899] 10.222.0.57 () {56 vars in 1122 bytes} [Fri Jul 1 16:28:13 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:28:13.882654 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602bdb14-19ed-4124-8c67-90a2c7a96615 None admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:28:13.942619 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 442/900] 10.222.0.57 () {58 vars in 1023 bytes} [Fri Jul 1 16:28:13 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 01 16:28:13.942619 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e11ae1-4900-43ed-a824-7855afa3b230 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:28:14.013986 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602bdb14-19ed-4124-8c67-90a2c7a96615 None admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:28:14.056968 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459de505-583c-4bcf-b80b-77948dc18c11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:28:14.057768 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459de505-583c-4bcf-b80b-77948dc18c11 None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:28:14.058052 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459de505-583c-4bcf-b80b-77948dc18c11 None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:28:14.058330 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459de505-583c-4bcf-b80b-77948dc18c11 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:28:14.061685 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e11ae1-4900-43ed-a824-7855afa3b230 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:28:14.065684 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-459de505-583c-4bcf-b80b-77948dc18c11 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:28:14.067737 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-459de505-583c-4bcf-b80b-77948dc18c11 None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:28:14.084177 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application [None req-459de505-583c-4bcf-b80b-77948dc18c11 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 01 16:28:14.084177 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 16:28:14.084177 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 16:28:14.084177 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 16:28:14.084177 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 16:28:14.084177 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 16:28:14.084177 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 16:28:14.084177 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 16:28:14.084177 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 16:28:14.084177 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 16:28:14.084177 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 16:28:14.084177 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 16:28:14.084177 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 01 16:28:14.084177 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 01 16:28:14.084177 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 01 16:28:14.085142 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 01 16:28:14.085142 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 16:28:14.085142 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 16:28:14.085142 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 16:28:14.085142 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 16:28:14.085142 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 16:28:14.085142 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 16:28:14.085142 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 16:28:14.085142 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application with Lock( Jul 01 16:28:14.085142 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 16:28:14.085142 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self._enter() Jul 01 16:28:14.085142 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 16:28:14.085142 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 16:28:14.085142 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 16:28:14.085142 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.creator() Jul 01 16:28:14.085142 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 16:28:14.085142 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application created_value = creator( Jul 01 16:28:14.086011 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 01 16:28:14.086011 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 01 16:28:14.086011 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 01 16:28:14.086011 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 01 16:28:14.086011 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 01 16:28:14.086011 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 01 16:28:14.086011 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 01 16:28:14.086011 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application Jul 01 16:28:14.086011 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 459/901] 10.222.0.57 () {56 vars in 1111 bytes} [Fri Jul 1 16:28:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 16:28:14.087638 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459de505-583c-4bcf-b80b-77948dc18c11 None admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:28:14.105147 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45decf2c-c586-47ea-b564-584570a8bf1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:28:14.105973 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45decf2c-c586-47ea-b564-584570a8bf1d None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:28:14.106276 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45decf2c-c586-47ea-b564-584570a8bf1d None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:28:14.106590 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45decf2c-c586-47ea-b564-584570a8bf1d None admin] PATH_INFO: `/v3/services` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:28:14.107338 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45decf2c-c586-47ea-b564-584570a8bf1d None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:28:14.107651 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45decf2c-c586-47ea-b564-584570a8bf1d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:28:14.109041 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45decf2c-c586-47ea-b564-584570a8bf1d None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:28:14.111077 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:28:14.115932 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 443/902] 10.222.0.57 () {56 vars in 1122 bytes} [Fri Jul 1 16:28:13 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:28:14.117205 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45decf2c-c586-47ea-b564-584570a8bf1d None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:28:14.136164 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=100 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:28:14.231878 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459de505-583c-4bcf-b80b-77948dc18c11 None admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:28:14.237641 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45decf2c-c586-47ea-b564-584570a8bf1d None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:28:14.277966 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321a92fe-4166-442a-80b9-d18a0da3d3a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:28:14.279261 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321a92fe-4166-442a-80b9-d18a0da3d3a0 None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:28:14.279749 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321a92fe-4166-442a-80b9-d18a0da3d3a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:28:14.280249 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97b0a3f-b4f5-453c-a784-5e1ea36eb1f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:28:14.280457 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321a92fe-4166-442a-80b9-d18a0da3d3a0 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:28:14.281365 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321a92fe-4166-442a-80b9-d18a0da3d3a0 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:28:14.281909 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97b0a3f-b4f5-453c-a784-5e1ea36eb1f5 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:28:14.282708 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97b0a3f-b4f5-453c-a784-5e1ea36eb1f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:28:14.283330 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97b0a3f-b4f5-453c-a784-5e1ea36eb1f5 None admin] PATH_INFO: `/v3/projects` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:28:14.283701 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321a92fe-4166-442a-80b9-d18a0da3d3a0 None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:28:14.285719 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c97b0a3f-b4f5-453c-a784-5e1ea36eb1f5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:28:14.288114 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c97b0a3f-b4f5-453c-a784-5e1ea36eb1f5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:28:14.289652 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c97b0a3f-b4f5-453c-a784-5e1ea36eb1f5 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:28:14.298514 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application [None req-321a92fe-4166-442a-80b9-d18a0da3d3a0 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 01 16:28:14.298514 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 16:28:14.298514 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 01 16:28:14.298514 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 01 16:28:14.298514 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 01 16:28:14.298514 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 01 16:28:14.298514 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 01 16:28:14.298514 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 01 16:28:14.298514 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 01 16:28:14.298514 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application Jul 01 16:28:14.298514 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 01 16:28:14.298514 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application Jul 01 16:28:14.298514 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 16:28:14.298514 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 16:28:14.298514 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 16:28:14.298514 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 16:28:14.299526 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 16:28:14.299526 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 16:28:14.299526 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 16:28:14.299526 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 16:28:14.299526 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 16:28:14.299526 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 16:28:14.299526 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 16:28:14.299526 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 01 16:28:14.299526 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 01 16:28:14.299526 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 01 16:28:14.299526 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 01 16:28:14.299526 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 16:28:14.299526 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 16:28:14.299526 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 16:28:14.299526 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 16:28:14.299526 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 16:28:14.300391 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 16:28:14.300391 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 16:28:14.300391 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application with Lock( Jul 01 16:28:14.300391 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 16:28:14.300391 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self._enter() Jul 01 16:28:14.300391 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 16:28:14.300391 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 16:28:14.300391 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 16:28:14.300391 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.creator() Jul 01 16:28:14.300391 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 16:28:14.300391 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application created_value = creator( Jul 01 16:28:14.300391 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 01 16:28:14.300391 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 01 16:28:14.300391 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 01 16:28:14.300391 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application Jul 01 16:28:14.304606 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 460/904] 10.222.0.57 () {56 vars in 1132 bytes} [Fri Jul 1 16:28:14 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 16:28:14.306389 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 444/904] 10.222.0.57 () {56 vars in 1148 bytes} [Fri Jul 1 16:28:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:28:14.310716 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321a92fe-4166-442a-80b9-d18a0da3d3a0 None admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:28:14.311694 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97b0a3f-b4f5-453c-a784-5e1ea36eb1f5 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:28:14.394741 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:28:14.418702 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=100 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:28:14.423037 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321a92fe-4166-442a-80b9-d18a0da3d3a0 None admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:28:14.452063 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97b0a3f-b4f5-453c-a784-5e1ea36eb1f5 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:28:14.470592 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f09c43-4c01-4adb-8da7-4d66c613d842 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:28:14.472118 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f09c43-4c01-4adb-8da7-4d66c613d842 None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:28:14.472585 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f09c43-4c01-4adb-8da7-4d66c613d842 None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:28:14.473302 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f09c43-4c01-4adb-8da7-4d66c613d842 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:28:14.474386 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f09c43-4c01-4adb-8da7-4d66c613d842 None admin] RBAC: Adding query filter params (role.id=c3a62bbbb749474aad6655e693c8963d, scope.project.id=e0788e6725a84e57a4174284c1845755, user.id=c4de1b1eeaed4fc5ae7200b2c7a90957) {{(pid=80312) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:28:14.474885 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f09c43-4c01-4adb-8da7-4d66c613d842 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:28:14.477179 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f09c43-4c01-4adb-8da7-4d66c613d842 None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:28:14.485945 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 461/905] 10.222.0.57 () {56 vars in 1374 bytes} [Fri Jul 1 16:28:14 2022] GET /identity/v3/role_assignments?user.id=c4de1b1eeaed4fc5ae7200b2c7a90957&scope.project.id=e0788e6725a84e57a4174284c1845755&role.id=c3a62bbbb749474aad6655e693c8963d => generated 259 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:28:14.500298 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4554b4e-8061-4745-96ea-a120c9a67afe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:28:14.501150 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4554b4e-8061-4745-96ea-a120c9a67afe None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:28:14.501547 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4554b4e-8061-4745-96ea-a120c9a67afe None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:28:14.501898 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4554b4e-8061-4745-96ea-a120c9a67afe None admin] PATH_INFO: `/v3/services` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:28:14.502850 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4554b4e-8061-4745-96ea-a120c9a67afe None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:28:14.503231 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4554b4e-8061-4745-96ea-a120c9a67afe None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:28:14.504879 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4554b4e-8061-4745-96ea-a120c9a67afe None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:28:14.511508 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 445/906] 10.222.0.57 () {56 vars in 1126 bytes} [Fri Jul 1 16:28:14 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:28:14.517970 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f09c43-4c01-4adb-8da7-4d66c613d842 None admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:28:14.640529 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f09c43-4c01-4adb-8da7-4d66c613d842 None admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:28:14.679815 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062e03b9-cc86-41e5-ae7a-556df2be53e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:28:14.680495 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062e03b9-cc86-41e5-ae7a-556df2be53e9 None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:28:14.680785 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062e03b9-cc86-41e5-ae7a-556df2be53e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:28:14.681069 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062e03b9-cc86-41e5-ae7a-556df2be53e9 None admin] PATH_INFO: `/v3/services` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:28:14.681791 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-062e03b9-cc86-41e5-ae7a-556df2be53e9 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80312) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:28:14.682109 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-062e03b9-cc86-41e5-ae7a-556df2be53e9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:28:14.683542 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-062e03b9-cc86-41e5-ae7a-556df2be53e9 None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:28:14.690047 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 462/907] 10.222.0.57 () {56 vars in 1126 bytes} [Fri Jul 1 16:28:14 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:28:14.697155 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4554b4e-8061-4745-96ea-a120c9a67afe None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:28:14.823961 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4554b4e-8061-4745-96ea-a120c9a67afe None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:28:14.851762 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:28:14.866839 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9ce4d8-7c1c-4545-b6e2-d9bbfdb11fdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:28:14.867606 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9ce4d8-7c1c-4545-b6e2-d9bbfdb11fdb None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:28:14.867896 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9ce4d8-7c1c-4545-b6e2-d9bbfdb11fdb None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:28:14.868238 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9ce4d8-7c1c-4545-b6e2-d9bbfdb11fdb None admin] PATH_INFO: `/v3/endpoints` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:28:14.869046 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a9ce4d8-7c1c-4545-b6e2-d9bbfdb11fdb None admin] RBAC: Adding query filter params (interface=public, service_id=9ede647cf7834f54b624f84aa100a90a, region_id=RegionOne) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:28:14.869410 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a9ce4d8-7c1c-4545-b6e2-d9bbfdb11fdb None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:28:14.870817 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a9ce4d8-7c1c-4545-b6e2-d9bbfdb11fdb None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:28:14.877184 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 446/908] 10.222.0.57 () {56 vars in 1251 bytes} [Fri Jul 1 16:28:14 2022] GET /identity/v3/endpoints?service_id=9ede647cf7834f54b624f84aa100a90a&interface=public®ion_id=RegionOne => generated 194 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:28:14.883082 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062e03b9-cc86-41e5-ae7a-556df2be53e9 None admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:28:14.915263 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=100 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:28:14.997636 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062e03b9-cc86-41e5-ae7a-556df2be53e9 None admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:28:15.034509 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc37ac3-bff9-4eda-939d-6f697f77c90c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:28:15.035183 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc37ac3-bff9-4eda-939d-6f697f77c90c None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:28:15.035471 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc37ac3-bff9-4eda-939d-6f697f77c90c None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:28:15.035810 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc37ac3-bff9-4eda-939d-6f697f77c90c None admin] PATH_INFO: `/v3/services` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:28:15.036893 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc37ac3-bff9-4eda-939d-6f697f77c90c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:28:15.038280 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc37ac3-bff9-4eda-939d-6f697f77c90c None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:28:15.044745 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 463/909] 10.222.0.57 () {56 vars in 1087 bytes} [Fri Jul 1 16:28:14 2022] GET /identity/v3/services => generated 1268 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 16:28:15.974491 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1123f6e1-a49f-42af-9924-fb170ba421aa None None] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:28:15.975500 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1123f6e1-a49f-42af-9924-fb170ba421aa None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:28:15.976004 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1123f6e1-a49f-42af-9924-fb170ba421aa None None] PATH_INFO: `/` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:28:15.977829 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 447/910] 10.222.0.57 () {54 vars in 926 bytes} [Fri Jul 1 16:28:15 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 16:28:15.983757 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43fcfe5-5d8d-4ba8-b312-8f825d51e8d4 None None] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:28:15.984580 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43fcfe5-5d8d-4ba8-b312-8f825d51e8d4 None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:28:15.985062 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43fcfe5-5d8d-4ba8-b312-8f825d51e8d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:28:16.003098 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.auth.core [None req-b43fcfe5-5d8d-4ba8-b312-8f825d51e8d4 None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80312) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:28:16.004759 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-b43fcfe5-5d8d-4ba8-b312-8f825d51e8d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:28:16.076521 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 464/911] 10.222.0.57 () {58 vars in 1023 bytes} [Fri Jul 1 16:28:15 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 01 16:28:16.102079 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1123f6e1-a49f-42af-9924-fb170ba421aa None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:28:16.231864 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1123f6e1-a49f-42af-9924-fb170ba421aa None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:28:16.273693 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785ae350-2d8c-42fa-8d25-4f1b5ef6d733 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:28:16.274538 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785ae350-2d8c-42fa-8d25-4f1b5ef6d733 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:28:16.274822 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785ae350-2d8c-42fa-8d25-4f1b5ef6d733 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:28:16.275096 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785ae350-2d8c-42fa-8d25-4f1b5ef6d733 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:28:16.275748 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785ae350-2d8c-42fa-8d25-4f1b5ef6d733 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:28:16.277118 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785ae350-2d8c-42fa-8d25-4f1b5ef6d733 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:28:16.290091 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application [None req-785ae350-2d8c-42fa-8d25-4f1b5ef6d733 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 01 16:28:16.290091 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 16:28:16.290091 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 01 16:28:16.290091 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 01 16:28:16.290091 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 01 16:28:16.290091 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 01 16:28:16.290091 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 01 16:28:16.290091 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 01 16:28:16.290091 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 01 16:28:16.290091 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Jul 01 16:28:16.290091 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 01 16:28:16.290091 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Jul 01 16:28:16.290091 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 16:28:16.290091 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 16:28:16.290091 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 16:28:16.290091 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 16:28:16.290091 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 16:28:16.291155 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 16:28:16.291155 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 16:28:16.291155 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 16:28:16.291155 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 16:28:16.291155 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 16:28:16.291155 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 16:28:16.291155 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 01 16:28:16.291155 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 01 16:28:16.291155 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 01 16:28:16.291155 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 01 16:28:16.291155 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 16:28:16.291155 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 16:28:16.291155 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 16:28:16.291155 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 16:28:16.291155 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 16:28:16.291155 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 16:28:16.291989 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 16:28:16.291989 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application with Lock( Jul 01 16:28:16.291989 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 16:28:16.291989 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._enter() Jul 01 16:28:16.291989 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 16:28:16.291989 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 16:28:16.291989 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 16:28:16.291989 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.creator() Jul 01 16:28:16.291989 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 16:28:16.291989 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application created_value = creator( Jul 01 16:28:16.291989 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 01 16:28:16.291989 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 01 16:28:16.291989 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 01 16:28:16.291989 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Jul 01 16:28:16.291989 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 448/912] 10.222.0.57 () {56 vars in 1114 bytes} [Fri Jul 1 16:28:16 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 16:28:16.299259 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43fcfe5-5d8d-4ba8-b312-8f825d51e8d4 None None] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:28:16.322815 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=100 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:28:16.331760 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa753aa-2386-4971-b747-2eb5a7d57313 None None] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:28:16.331760 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa753aa-2386-4971-b747-2eb5a7d57313 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:28:16.332040 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa753aa-2386-4971-b747-2eb5a7d57313 None None] PATH_INFO: `/` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:28:16.335003 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 449/913] 10.222.0.57 () {54 vars in 926 bytes} [Fri Jul 1 16:28:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 16:28:16.339301 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f115060-6927-4a5a-af95-1bd922ac5fb8 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:28:16.339589 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f115060-6927-4a5a-af95-1bd922ac5fb8 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:28:16.339868 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f115060-6927-4a5a-af95-1bd922ac5fb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:28:16.357512 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-1f115060-6927-4a5a-af95-1bd922ac5fb8 None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:28:16.358648 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-1f115060-6927-4a5a-af95-1bd922ac5fb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:28:16.411276 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:28:16.413848 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43fcfe5-5d8d-4ba8-b312-8f825d51e8d4 None None] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:28:16.438468 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 450/914] 10.222.0.57 () {58 vars in 1023 bytes} [Fri Jul 1 16:28:16 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 01 16:28:16.451939 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ea34ad-45fd-4226-95ca-e53c4c6d66a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:28:16.453200 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ea34ad-45fd-4226-95ca-e53c4c6d66a9 None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:28:16.453685 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ea34ad-45fd-4226-95ca-e53c4c6d66a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:28:16.454168 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ea34ad-45fd-4226-95ca-e53c4c6d66a9 None admin] PATH_INFO: `/v3/services` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:28:16.455213 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ea34ad-45fd-4226-95ca-e53c4c6d66a9 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80312) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:28:16.455757 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ea34ad-45fd-4226-95ca-e53c4c6d66a9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:28:16.455954 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f115060-6927-4a5a-af95-1bd922ac5fb8 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:28:16.458250 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ea34ad-45fd-4226-95ca-e53c4c6d66a9 None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:28:16.464324 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 465/915] 10.222.0.57 () {56 vars in 1114 bytes} [Fri Jul 1 16:28:16 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:28:16.470002 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ea34ad-45fd-4226-95ca-e53c4c6d66a9 None admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:28:16.573956 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f115060-6927-4a5a-af95-1bd922ac5fb8 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:28:16.590756 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=100 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:28:16.615109 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8092d8d-d59a-48c0-9a6a-0b3efd4f853c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:28:16.615947 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8092d8d-d59a-48c0-9a6a-0b3efd4f853c None admin] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:28:16.616722 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8092d8d-d59a-48c0-9a6a-0b3efd4f853c None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:28:16.617078 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8092d8d-d59a-48c0-9a6a-0b3efd4f853c None admin] PATH_INFO: `/v3/services` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:28:16.618372 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ea34ad-45fd-4226-95ca-e53c4c6d66a9 None admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:28:16.618625 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8092d8d-d59a-48c0-9a6a-0b3efd4f853c None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:28:16.620084 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8092d8d-d59a-48c0-9a6a-0b3efd4f853c None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:28:16.643535 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 451/916] 10.222.0.57 () {60 vars in 1141 bytes} [Fri Jul 1 16:28:16 2022] POST /identity/v3/services => generated 241 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 01 16:28:16.669565 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb4659d-08ab-41ec-aea2-e8f806d9230f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:28:16.670652 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb4659d-08ab-41ec-aea2-e8f806d9230f None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:28:16.671124 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb4659d-08ab-41ec-aea2-e8f806d9230f None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:28:16.671584 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb4659d-08ab-41ec-aea2-e8f806d9230f None admin] PATH_INFO: `/v3/services` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:28:16.672508 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb4659d-08ab-41ec-aea2-e8f806d9230f None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80312) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:28:16.673065 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb4659d-08ab-41ec-aea2-e8f806d9230f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:28:16.674801 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb4659d-08ab-41ec-aea2-e8f806d9230f None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:28:16.678800 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117b98ab-b554-408c-a751-7fd9e2a450bb None None] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:28:16.679093 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117b98ab-b554-408c-a751-7fd9e2a450bb None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:28:16.679484 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117b98ab-b554-408c-a751-7fd9e2a450bb None None] PATH_INFO: `/` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:28:16.682400 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 452/918] 10.222.0.57 () {54 vars in 926 bytes} [Fri Jul 1 16:28:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 16:28:16.682549 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 466/918] 10.222.0.57 () {56 vars in 1114 bytes} [Fri Jul 1 16:28:16 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 01 16:28:16.686675 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33259aeb-60d8-407a-930f-120be80ac571 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:28:16.687281 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33259aeb-60d8-407a-930f-120be80ac571 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:28:16.687795 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33259aeb-60d8-407a-930f-120be80ac571 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:28:16.688124 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb4659d-08ab-41ec-aea2-e8f806d9230f None admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:28:16.713505 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-33259aeb-60d8-407a-930f-120be80ac571 None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:28:16.715201 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-33259aeb-60d8-407a-930f-120be80ac571 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:28:16.766033 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:28:16.834847 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 453/919] 10.222.0.57 () {58 vars in 1023 bytes} [Fri Jul 1 16:28:16 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 01 16:28:16.843759 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb4659d-08ab-41ec-aea2-e8f806d9230f None admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:28:16.863485 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33259aeb-60d8-407a-930f-120be80ac571 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:28:16.889643 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a21429-d15f-4320-9d5b-92088acbec01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:28:16.890277 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a21429-d15f-4320-9d5b-92088acbec01 None admin] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:28:16.891487 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a21429-d15f-4320-9d5b-92088acbec01 None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:28:16.891775 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a21429-d15f-4320-9d5b-92088acbec01 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:28:16.892559 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a21429-d15f-4320-9d5b-92088acbec01 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:28:16.893952 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a21429-d15f-4320-9d5b-92088acbec01 None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:28:16.911495 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:28:16.923927 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 467/920] 10.222.0.57 () {60 vars in 1144 bytes} [Fri Jul 1 16:28:16 2022] POST /identity/v3/endpoints => generated 346 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 01 16:28:17.005204 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33259aeb-60d8-407a-930f-120be80ac571 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:28:17.055777 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98c0d92-8ee9-471c-acd0-929d223a738c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:28:17.056611 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98c0d92-8ee9-471c-acd0-929d223a738c None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:28:17.056888 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98c0d92-8ee9-471c-acd0-929d223a738c None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:28:17.057161 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98c0d92-8ee9-471c-acd0-929d223a738c None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:28:17.070627 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a98c0d92-8ee9-471c-acd0-929d223a738c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 01 16:28:17.080072 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application [None req-a98c0d92-8ee9-471c-acd0-929d223a738c None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 01 16:28:17.080072 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 16:28:17.080072 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 16:28:17.080072 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 16:28:17.080072 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 16:28:17.080072 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 16:28:17.080072 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 16:28:17.080072 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 16:28:17.080072 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 16:28:17.080072 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 16:28:17.080072 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 16:28:17.080072 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 16:28:17.080072 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 01 16:28:17.080072 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 01 16:28:17.080072 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 01 16:28:17.080072 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 01 16:28:17.081227 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 01 16:28:17.081227 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 01 16:28:17.081227 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 01 16:28:17.081227 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 01 16:28:17.081227 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 16:28:17.081227 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 16:28:17.081227 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 16:28:17.081227 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 16:28:17.081227 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 16:28:17.081227 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 16:28:17.081227 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 16:28:17.081227 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application with Lock( Jul 01 16:28:17.081227 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 16:28:17.081227 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._enter() Jul 01 16:28:17.081227 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 16:28:17.081227 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 16:28:17.082238 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 16:28:17.082238 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.creator() Jul 01 16:28:17.082238 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 16:28:17.082238 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application created_value = creator( Jul 01 16:28:17.082238 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 01 16:28:17.082238 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 01 16:28:17.082238 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 01 16:28:17.082238 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 01 16:28:17.082238 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 01 16:28:17.082238 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 01 16:28:17.082238 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 01 16:28:17.082238 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Jul 01 16:28:17.082238 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 454/921] 10.222.0.57 () {56 vars in 1120 bytes} [Fri Jul 1 16:28:16 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 16:28:17.090725 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a21429-d15f-4320-9d5b-92088acbec01 None admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:28:17.212733 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a21429-d15f-4320-9d5b-92088acbec01 None admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:28:17.248891 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c7068f-6e41-42a5-a4e5-0fcb311e5f4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:28:17.249620 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c7068f-6e41-42a5-a4e5-0fcb311e5f4d None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:28:17.249904 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c7068f-6e41-42a5-a4e5-0fcb311e5f4d None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:28:17.250179 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c7068f-6e41-42a5-a4e5-0fcb311e5f4d None admin] PATH_INFO: `/v3/roles` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:28:17.250962 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c7068f-6e41-42a5-a4e5-0fcb311e5f4d None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80312) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:28:17.251272 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c7068f-6e41-42a5-a4e5-0fcb311e5f4d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:28:17.252646 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c7068f-6e41-42a5-a4e5-0fcb311e5f4d None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:28:17.258683 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2bbc13-e4ba-4765-b8bf-751bc163d277 None None] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:28:17.258977 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2bbc13-e4ba-4765-b8bf-751bc163d277 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:28:17.259259 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2bbc13-e4ba-4765-b8bf-751bc163d277 None None] PATH_INFO: `/` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:28:17.260782 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 455/922] 10.222.0.57 () {54 vars in 926 bytes} [Fri Jul 1 16:28:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 16:28:17.263508 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 468/923] 10.222.0.57 () {56 vars in 1115 bytes} [Fri Jul 1 16:28:17 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:28:17.265881 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be75b08-65c2-497d-b075-890385f1cd26 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:28:17.266169 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be75b08-65c2-497d-b075-890385f1cd26 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:28:17.268303 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be75b08-65c2-497d-b075-890385f1cd26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:28:17.269756 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c7068f-6e41-42a5-a4e5-0fcb311e5f4d None admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:28:17.289241 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-5be75b08-65c2-497d-b075-890385f1cd26 None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:28:17.291229 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-5be75b08-65c2-497d-b075-890385f1cd26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:28:17.318461 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:28:17.341250 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=100 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:28:17.386645 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 456/924] 10.222.0.57 () {58 vars in 1023 bytes} [Fri Jul 1 16:28:17 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 01 16:28:17.388558 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c7068f-6e41-42a5-a4e5-0fcb311e5f4d None admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:28:17.408424 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be75b08-65c2-497d-b075-890385f1cd26 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:28:17.427045 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601acb3f-3630-4acf-a859-cc9410d30e1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:28:17.428436 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601acb3f-3630-4acf-a859-cc9410d30e1e None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:28:17.428948 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601acb3f-3630-4acf-a859-cc9410d30e1e None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:28:17.429534 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601acb3f-3630-4acf-a859-cc9410d30e1e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:28:17.441519 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-601acb3f-3630-4acf-a859-cc9410d30e1e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:28:17.443858 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-601acb3f-3630-4acf-a859-cc9410d30e1e None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:28:17.463417 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application [None req-601acb3f-3630-4acf-a859-cc9410d30e1e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 16:28:17.463417 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 16:28:17.463417 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 16:28:17.463417 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 16:28:17.463417 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 16:28:17.463417 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 16:28:17.463417 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 16:28:17.463417 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 16:28:17.463417 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 16:28:17.463417 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 16:28:17.463417 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 16:28:17.463417 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 16:28:17.463417 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 01 16:28:17.463417 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 01 16:28:17.463417 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 01 16:28:17.464567 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 01 16:28:17.464567 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 16:28:17.464567 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 16:28:17.464567 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 16:28:17.464567 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 16:28:17.464567 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 16:28:17.464567 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 16:28:17.464567 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 16:28:17.464567 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application with Lock( Jul 01 16:28:17.464567 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 16:28:17.464567 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self._enter() Jul 01 16:28:17.464567 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 16:28:17.464567 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 16:28:17.464567 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 16:28:17.464567 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.creator() Jul 01 16:28:17.464567 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 16:28:17.464567 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application created_value = creator( Jul 01 16:28:17.465527 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 01 16:28:17.465527 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 01 16:28:17.465527 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 16:28:17.465527 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application Jul 01 16:28:17.470492 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 469/925] 10.222.0.57 () {56 vars in 1108 bytes} [Fri Jul 1 16:28:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 01 16:28:17.472034 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601acb3f-3630-4acf-a859-cc9410d30e1e None admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:28:17.552789 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be75b08-65c2-497d-b075-890385f1cd26 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:28:17.591167 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601acb3f-3630-4acf-a859-cc9410d30e1e None admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:28:17.595529 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07dc57df-2ddc-4cab-9366-385079a87f2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:28:17.596642 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07dc57df-2ddc-4cab-9366-385079a87f2c None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:28:17.597130 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07dc57df-2ddc-4cab-9366-385079a87f2c None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:28:17.597605 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07dc57df-2ddc-4cab-9366-385079a87f2c None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:28:17.599426 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07dc57df-2ddc-4cab-9366-385079a87f2c None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:28:17.601128 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07dc57df-2ddc-4cab-9366-385079a87f2c None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:28:17.606238 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=100 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:28:17.617747 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application [None req-07dc57df-2ddc-4cab-9366-385079a87f2c None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 01 16:28:17.617747 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 16:28:17.617747 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 01 16:28:17.617747 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 01 16:28:17.617747 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 01 16:28:17.617747 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 01 16:28:17.617747 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 01 16:28:17.617747 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 01 16:28:17.617747 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 01 16:28:17.617747 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Jul 01 16:28:17.617747 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 01 16:28:17.617747 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Jul 01 16:28:17.617747 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 16:28:17.617747 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 16:28:17.617747 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 16:28:17.617747 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 16:28:17.618950 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 16:28:17.618950 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 16:28:17.618950 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 16:28:17.618950 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 16:28:17.618950 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 16:28:17.618950 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 16:28:17.618950 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 16:28:17.618950 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 01 16:28:17.618950 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 01 16:28:17.618950 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 01 16:28:17.618950 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 01 16:28:17.618950 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 16:28:17.618950 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 16:28:17.618950 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 16:28:17.618950 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 16:28:17.618950 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 16:28:17.619796 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 16:28:17.619796 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 16:28:17.619796 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application with Lock( Jul 01 16:28:17.619796 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 16:28:17.619796 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._enter() Jul 01 16:28:17.619796 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 16:28:17.619796 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 16:28:17.619796 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 16:28:17.619796 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.creator() Jul 01 16:28:17.619796 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 16:28:17.619796 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application created_value = creator( Jul 01 16:28:17.619796 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 01 16:28:17.619796 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 01 16:28:17.619796 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 01 16:28:17.619796 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Jul 01 16:28:17.627187 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 457/926] 10.222.0.57 () {56 vars in 1132 bytes} [Fri Jul 1 16:28:17 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 01 16:28:17.630175 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07dc57df-2ddc-4cab-9366-385079a87f2c None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:28:17.631300 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f8d140-6ad5-4ee6-96f2-3f5c8bdf1ae2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:28:17.631982 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f8d140-6ad5-4ee6-96f2-3f5c8bdf1ae2 None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:28:17.632282 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f8d140-6ad5-4ee6-96f2-3f5c8bdf1ae2 None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:28:17.632565 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f8d140-6ad5-4ee6-96f2-3f5c8bdf1ae2 None admin] PATH_INFO: `/v3/domains` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:28:17.633302 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87f8d140-6ad5-4ee6-96f2-3f5c8bdf1ae2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80312) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:28:17.633614 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87f8d140-6ad5-4ee6-96f2-3f5c8bdf1ae2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:28:17.635165 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87f8d140-6ad5-4ee6-96f2-3f5c8bdf1ae2 None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:28:17.653921 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 470/927] 10.222.0.57 () {56 vars in 1109 bytes} [Fri Jul 1 16:28:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 01 16:28:17.655212 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f8d140-6ad5-4ee6-96f2-3f5c8bdf1ae2 None admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:28:17.658693 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:28:17.747899 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07dc57df-2ddc-4cab-9366-385079a87f2c None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:28:17.762962 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f8d140-6ad5-4ee6-96f2-3f5c8bdf1ae2 None admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:28:17.788306 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e71e42-b7bd-45f9-8453-c06e5edcee13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:28:17.789464 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e71e42-b7bd-45f9-8453-c06e5edcee13 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:28:17.789989 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e71e42-b7bd-45f9-8453-c06e5edcee13 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:28:17.790543 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e71e42-b7bd-45f9-8453-c06e5edcee13 None admin] PATH_INFO: `/v3/services` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:28:17.791581 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e71e42-b7bd-45f9-8453-c06e5edcee13 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:28:17.792096 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e71e42-b7bd-45f9-8453-c06e5edcee13 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:28:17.793736 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e71e42-b7bd-45f9-8453-c06e5edcee13 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:28:17.805286 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2470b7e-76c8-432f-9f97-ccaab820edb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:28:17.805286 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2470b7e-76c8-432f-9f97-ccaab820edb1 None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:28:17.805286 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2470b7e-76c8-432f-9f97-ccaab820edb1 None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:28:17.805286 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2470b7e-76c8-432f-9f97-ccaab820edb1 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:28:17.806185 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 458/928] 10.222.0.57 () {56 vars in 1126 bytes} [Fri Jul 1 16:28:17 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 01 16:28:17.808052 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e71e42-b7bd-45f9-8453-c06e5edcee13 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:28:17.813522 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2470b7e-76c8-432f-9f97-ccaab820edb1 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:28:17.815016 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2470b7e-76c8-432f-9f97-ccaab820edb1 None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:28:17.832515 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application [None req-b2470b7e-76c8-432f-9f97-ccaab820edb1 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 01 16:28:17.832515 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 16:28:17.832515 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 16:28:17.832515 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 16:28:17.832515 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 16:28:17.832515 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 16:28:17.832515 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 16:28:17.832515 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 16:28:17.832515 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 16:28:17.832515 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 16:28:17.832515 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 16:28:17.832515 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 16:28:17.832515 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 01 16:28:17.832515 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 01 16:28:17.832515 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 01 16:28:17.832515 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 01 16:28:17.833912 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 16:28:17.833912 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 16:28:17.833912 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 01 16:28:17.833912 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 01 16:28:17.833912 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 01 16:28:17.833912 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 01 16:28:17.833912 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 16:28:17.833912 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 16:28:17.833912 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 16:28:17.833912 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 16:28:17.833912 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 16:28:17.833912 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application with Lock( Jul 01 16:28:17.833912 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 16:28:17.833912 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self._enter() Jul 01 16:28:17.833912 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 16:28:17.833912 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 16:28:17.833912 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 16:28:17.833912 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.creator() Jul 01 16:28:17.834887 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 16:28:17.834887 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application created_value = creator( Jul 01 16:28:17.834887 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 01 16:28:17.834887 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 01 16:28:17.834887 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 01 16:28:17.834887 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 01 16:28:17.834887 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 01 16:28:17.834887 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 01 16:28:17.834887 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 01 16:28:17.834887 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application Jul 01 16:28:17.838466 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 471/929] 10.222.0.57 () {56 vars in 1117 bytes} [Fri Jul 1 16:28:17 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 01 16:28:17.839884 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2470b7e-76c8-432f-9f97-ccaab820edb1 None admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:28:17.873056 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=100 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:28:17.914725 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:28:17.934924 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e71e42-b7bd-45f9-8453-c06e5edcee13 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:28:17.948790 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2470b7e-76c8-432f-9f97-ccaab820edb1 None admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:28:17.980488 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15aa27ba-c010-4590-be15-ab2a0306fe1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:28:17.981157 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15aa27ba-c010-4590-be15-ab2a0306fe1f None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:28:17.981442 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15aa27ba-c010-4590-be15-ab2a0306fe1f None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:28:17.981722 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15aa27ba-c010-4590-be15-ab2a0306fe1f None admin] PATH_INFO: `/v3/services` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:28:17.982469 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15aa27ba-c010-4590-be15-ab2a0306fe1f None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:28:17.982781 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15aa27ba-c010-4590-be15-ab2a0306fe1f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:28:17.984146 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15aa27ba-c010-4590-be15-ab2a0306fe1f None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:28:17.990412 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 459/930] 10.222.0.57 () {56 vars in 1126 bytes} [Fri Jul 1 16:28:17 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:28:17.996399 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15aa27ba-c010-4590-be15-ab2a0306fe1f None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:28:18.001389 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a2e0bf-36f1-487b-b99f-82f0a218489d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:28:18.002269 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a2e0bf-36f1-487b-b99f-82f0a218489d None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:28:18.002746 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a2e0bf-36f1-487b-b99f-82f0a218489d None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:28:18.003188 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a2e0bf-36f1-487b-b99f-82f0a218489d None admin] PATH_INFO: `/v3/users` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:28:18.004220 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a2e0bf-36f1-487b-b99f-82f0a218489d None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80312) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:28:18.004723 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a2e0bf-36f1-487b-b99f-82f0a218489d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:28:18.006339 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a2e0bf-36f1-487b-b99f-82f0a218489d None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:28:18.030396 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 472/931] 10.222.0.57 () {56 vars in 1149 bytes} [Fri Jul 1 16:28:17 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 01 16:28:18.032006 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a2e0bf-36f1-487b-b99f-82f0a218489d None admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:28:18.116460 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15aa27ba-c010-4590-be15-ab2a0306fe1f None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:28:18.158680 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=100 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:28:18.162786 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5162eebc-6469-4530-a2da-3d36c70a05bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:28:18.164640 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5162eebc-6469-4530-a2da-3d36c70a05bd None admin] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:28:18.165113 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5162eebc-6469-4530-a2da-3d36c70a05bd None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:28:18.165623 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5162eebc-6469-4530-a2da-3d36c70a05bd None admin] PATH_INFO: `/v3/endpoints` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:28:18.166618 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5162eebc-6469-4530-a2da-3d36c70a05bd None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:28:18.168438 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5162eebc-6469-4530-a2da-3d36c70a05bd None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:28:18.171216 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a2e0bf-36f1-487b-b99f-82f0a218489d None admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:28:18.194322 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 460/932] 10.222.0.57 () {60 vars in 1144 bytes} [Fri Jul 1 16:28:17 2022] POST /identity/v3/endpoints => generated 347 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 01 16:28:18.231974 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ce8175-4216-4bcb-91b8-43f23733e1cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:28:18.233460 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ce8175-4216-4bcb-91b8-43f23733e1cd None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:28:18.233953 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ce8175-4216-4bcb-91b8-43f23733e1cd None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:28:18.234552 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ce8175-4216-4bcb-91b8-43f23733e1cd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:28:18.248763 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ce8175-4216-4bcb-91b8-43f23733e1cd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:28:18.250950 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ce8175-4216-4bcb-91b8-43f23733e1cd None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:28:18.274478 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application [None req-18ce8175-4216-4bcb-91b8-43f23733e1cd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 16:28:18.274478 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 16:28:18.274478 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 16:28:18.274478 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 16:28:18.274478 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 16:28:18.274478 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 16:28:18.274478 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 16:28:18.274478 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 16:28:18.274478 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 16:28:18.274478 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 16:28:18.274478 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 16:28:18.274478 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 16:28:18.274478 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 01 16:28:18.274478 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 01 16:28:18.274478 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 01 16:28:18.275686 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 01 16:28:18.275686 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 16:28:18.275686 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 16:28:18.275686 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 16:28:18.275686 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 16:28:18.275686 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 16:28:18.275686 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 16:28:18.275686 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 16:28:18.275686 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application with Lock( Jul 01 16:28:18.275686 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 16:28:18.275686 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self._enter() Jul 01 16:28:18.275686 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 16:28:18.275686 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 16:28:18.275686 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 16:28:18.275686 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.creator() Jul 01 16:28:18.275686 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 16:28:18.275686 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application created_value = creator( Jul 01 16:28:18.276595 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 01 16:28:18.276595 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 01 16:28:18.276595 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 16:28:18.276595 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application Jul 01 16:28:18.284607 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 473/933] 10.222.0.57 () {56 vars in 1108 bytes} [Fri Jul 1 16:28:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 16:28:18.287163 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5162eebc-6469-4530-a2da-3d36c70a05bd None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:28:18.338295 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:28:18.412429 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5162eebc-6469-4530-a2da-3d36c70a05bd None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:28:18.452966 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7a83e2-9b70-42a2-9761-23a9dee0376c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:28:18.453804 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7a83e2-9b70-42a2-9761-23a9dee0376c None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:28:18.454311 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7a83e2-9b70-42a2-9761-23a9dee0376c None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:28:18.454771 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7a83e2-9b70-42a2-9761-23a9dee0376c None admin] PATH_INFO: `/v3/domains` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:28:18.455640 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7a83e2-9b70-42a2-9761-23a9dee0376c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:28:18.456050 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7a83e2-9b70-42a2-9761-23a9dee0376c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:28:18.457681 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7a83e2-9b70-42a2-9761-23a9dee0376c None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:28:18.474509 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 461/934] 10.222.0.57 () {56 vars in 1109 bytes} [Fri Jul 1 16:28:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:28:18.477294 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ce8175-4216-4bcb-91b8-43f23733e1cd None admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:28:18.620116 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ce8175-4216-4bcb-91b8-43f23733e1cd None admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:28:18.660935 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f711d7b5-99c8-4d76-97fc-e2a730d75c54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:28:18.662313 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f711d7b5-99c8-4d76-97fc-e2a730d75c54 None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:28:18.662836 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f711d7b5-99c8-4d76-97fc-e2a730d75c54 None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:28:18.663377 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f711d7b5-99c8-4d76-97fc-e2a730d75c54 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:28:18.671451 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f711d7b5-99c8-4d76-97fc-e2a730d75c54 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:28:18.673259 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f711d7b5-99c8-4d76-97fc-e2a730d75c54 None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:28:18.675376 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=100 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:28:18.693034 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application [None req-f711d7b5-99c8-4d76-97fc-e2a730d75c54 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 01 16:28:18.693034 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 16:28:18.693034 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 16:28:18.693034 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 16:28:18.693034 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 16:28:18.693034 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 16:28:18.693034 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 16:28:18.693034 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 16:28:18.693034 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 16:28:18.693034 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 16:28:18.693034 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 16:28:18.693034 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 16:28:18.693034 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 01 16:28:18.693034 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 01 16:28:18.693034 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 01 16:28:18.694023 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 01 16:28:18.694023 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 16:28:18.694023 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 16:28:18.694023 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 16:28:18.694023 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 16:28:18.694023 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 16:28:18.694023 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 16:28:18.694023 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 16:28:18.694023 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application with Lock( Jul 01 16:28:18.694023 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 16:28:18.694023 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self._enter() Jul 01 16:28:18.694023 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 16:28:18.694023 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 16:28:18.694023 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 16:28:18.694023 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.creator() Jul 01 16:28:18.694023 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 16:28:18.694023 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application created_value = creator( Jul 01 16:28:18.694954 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 01 16:28:18.694954 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 01 16:28:18.694954 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 01 16:28:18.694954 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 01 16:28:18.694954 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 01 16:28:18.694954 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 01 16:28:18.694954 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 01 16:28:18.694954 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application Jul 01 16:28:18.701965 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 474/935] 10.222.0.57 () {56 vars in 1111 bytes} [Fri Jul 1 16:28:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 16:28:18.703891 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7a83e2-9b70-42a2-9761-23a9dee0376c None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:28:18.776503 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:28:18.829358 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7a83e2-9b70-42a2-9761-23a9dee0376c None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:28:18.870908 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2db8ed-891f-442c-a987-28d1d0ba1832 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:28:18.871502 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2db8ed-891f-442c-a987-28d1d0ba1832 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:28:18.871788 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2db8ed-891f-442c-a987-28d1d0ba1832 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:28:18.872070 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2db8ed-891f-442c-a987-28d1d0ba1832 None admin] PATH_INFO: `/v3/projects` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:28:18.872871 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd2db8ed-891f-442c-a987-28d1d0ba1832 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:28:18.873230 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd2db8ed-891f-442c-a987-28d1d0ba1832 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:28:18.875344 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd2db8ed-891f-442c-a987-28d1d0ba1832 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:28:18.891610 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 462/936] 10.222.0.57 () {56 vars in 1148 bytes} [Fri Jul 1 16:28:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:28:18.897576 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f711d7b5-99c8-4d76-97fc-e2a730d75c54 None admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:28:19.011565 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f711d7b5-99c8-4d76-97fc-e2a730d75c54 None admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:28:19.038648 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6117ca1a-0551-4dae-9c15-11e9192a0eb8 None None] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:28:19.039199 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6117ca1a-0551-4dae-9c15-11e9192a0eb8 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:28:19.039631 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6117ca1a-0551-4dae-9c15-11e9192a0eb8 None None] PATH_INFO: `/` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:28:19.041417 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 463/937] 10.222.0.57 () {54 vars in 926 bytes} [Fri Jul 1 16:28:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 16:28:19.046918 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a53614-3023-48eb-9829-db2d16aad0e9 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:28:19.047403 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a53614-3023-48eb-9829-db2d16aad0e9 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:28:19.048101 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a53614-3023-48eb-9829-db2d16aad0e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:28:19.049507 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37dcec27-6ef4-4c35-8d86-b9592eb58a59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:28:19.050515 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37dcec27-6ef4-4c35-8d86-b9592eb58a59 None admin] REQUEST_METHOD: `PUT` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:28:19.050831 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37dcec27-6ef4-4c35-8d86-b9592eb58a59 None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:28:19.051148 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37dcec27-6ef4-4c35-8d86-b9592eb58a59 None admin] PATH_INFO: `/v3/projects/e0788e6725a84e57a4174284c1845755/users/c4de1b1eeaed4fc5ae7200b2c7a90957/roles/c3a62bbbb749474aad6655e693c8963d` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:28:19.053712 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37dcec27-6ef4-4c35-8d86-b9592eb58a59 None admin] RBAC: Authorizing `identity:create_grant(project_id=e0788e6725a84e57a4174284c1845755, user_id=c4de1b1eeaed4fc5ae7200b2c7a90957, role_id=c3a62bbbb749474aad6655e693c8963d)` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:28:19.055453 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37dcec27-6ef4-4c35-8d86-b9592eb58a59 None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:28:19.070769 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-b5a53614-3023-48eb-9829-db2d16aad0e9 None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:28:19.071987 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-b5a53614-3023-48eb-9829-db2d16aad0e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:28:19.073256 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 475/938] 10.222.0.57 () {58 vars in 1439 bytes} [Fri Jul 1 16:28:18 2022] PUT /identity/v3/projects/e0788e6725a84e57a4174284c1845755/users/c4de1b1eeaed4fc5ae7200b2c7a90957/roles/c3a62bbbb749474aad6655e693c8963d => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 16:28:19.171701 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 464/939] 10.222.0.57 () {58 vars in 1023 bytes} [Fri Jul 1 16:28:19 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 01 16:28:19.197912 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37dcec27-6ef4-4c35-8d86-b9592eb58a59 None admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:28:19.236487 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da2663b-8726-4d64-baec-1ebdba00dfbb None None] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:28:19.236821 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da2663b-8726-4d64-baec-1ebdba00dfbb None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:28:19.237057 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da2663b-8726-4d64-baec-1ebdba00dfbb None None] PATH_INFO: `/` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:28:19.238470 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 465/940] 10.222.0.57 () {54 vars in 926 bytes} [Fri Jul 1 16:28:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 16:28:19.245868 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b06ae98-04c7-4c36-b7f6-fa5432634729 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:28:19.246708 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b06ae98-04c7-4c36-b7f6-fa5432634729 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:28:19.248052 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b06ae98-04c7-4c36-b7f6-fa5432634729 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:28:19.267898 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-4b06ae98-04c7-4c36-b7f6-fa5432634729 None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:28:19.269093 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-4b06ae98-04c7-4c36-b7f6-fa5432634729 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:28:19.284222 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:28:19.336114 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37dcec27-6ef4-4c35-8d86-b9592eb58a59 None admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:28:19.362557 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:28:19.376444 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f02dc28-91c2-4213-9a4c-835d9307ed6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:28:19.377259 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f02dc28-91c2-4213-9a4c-835d9307ed6e None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:28:19.377545 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f02dc28-91c2-4213-9a4c-835d9307ed6e None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:28:19.377822 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f02dc28-91c2-4213-9a4c-835d9307ed6e None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:28:19.378524 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f02dc28-91c2-4213-9a4c-835d9307ed6e None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:28:19.380610 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f02dc28-91c2-4213-9a4c-835d9307ed6e None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:28:19.390664 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 466/941] 10.222.0.57 () {58 vars in 1023 bytes} [Fri Jul 1 16:28:19 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 01 16:28:19.401640 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application [None req-8f02dc28-91c2-4213-9a4c-835d9307ed6e None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 01 16:28:19.401640 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 16:28:19.401640 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 01 16:28:19.401640 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 01 16:28:19.401640 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 01 16:28:19.401640 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 01 16:28:19.401640 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 01 16:28:19.401640 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 01 16:28:19.401640 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 01 16:28:19.401640 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application Jul 01 16:28:19.401640 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 01 16:28:19.401640 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application Jul 01 16:28:19.401640 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 16:28:19.401640 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 16:28:19.401640 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 16:28:19.401640 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 16:28:19.401640 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 16:28:19.403208 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 16:28:19.403208 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 16:28:19.403208 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 16:28:19.403208 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 16:28:19.403208 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 16:28:19.403208 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 16:28:19.403208 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 01 16:28:19.403208 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 01 16:28:19.403208 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 01 16:28:19.403208 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 01 16:28:19.403208 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 16:28:19.403208 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 16:28:19.403208 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 16:28:19.403208 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 16:28:19.403208 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 16:28:19.403208 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 16:28:19.404275 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 16:28:19.404275 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application with Lock( Jul 01 16:28:19.404275 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 16:28:19.404275 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self._enter() Jul 01 16:28:19.404275 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 16:28:19.404275 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 16:28:19.404275 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 16:28:19.404275 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.creator() Jul 01 16:28:19.404275 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 16:28:19.404275 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application created_value = creator( Jul 01 16:28:19.404275 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 01 16:28:19.404275 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 01 16:28:19.404275 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 01 16:28:19.404275 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application Jul 01 16:28:19.404275 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 476/942] 10.222.0.57 () {56 vars in 1126 bytes} [Fri Jul 1 16:28:19 2022] GET /identity/v3/services/object-store => generated 93 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 16:28:19.405128 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b06ae98-04c7-4c36-b7f6-fa5432634729 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:28:19.414453 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f02dc28-91c2-4213-9a4c-835d9307ed6e None admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:28:19.542450 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b06ae98-04c7-4c36-b7f6-fa5432634729 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:28:19.544529 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f02dc28-91c2-4213-9a4c-835d9307ed6e None admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:28:19.579192 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=100 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:28:19.584127 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41afaa1-3c4b-4819-8276-936b480bdaa2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:28:19.584812 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41afaa1-3c4b-4819-8276-936b480bdaa2 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:28:19.585179 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41afaa1-3c4b-4819-8276-936b480bdaa2 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:28:19.585618 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41afaa1-3c4b-4819-8276-936b480bdaa2 None admin] PATH_INFO: `/v3/services` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:28:19.586541 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c41afaa1-3c4b-4819-8276-936b480bdaa2 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:28:19.586862 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c41afaa1-3c4b-4819-8276-936b480bdaa2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:28:19.588269 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c41afaa1-3c4b-4819-8276-936b480bdaa2 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:28:19.594555 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb5e384-21db-4ac9-a2d5-d2c6a48b7700 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:28:19.595229 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb5e384-21db-4ac9-a2d5-d2c6a48b7700 None admin] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:28:19.595515 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb5e384-21db-4ac9-a2d5-d2c6a48b7700 None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:28:19.595781 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb5e384-21db-4ac9-a2d5-d2c6a48b7700 None admin] PATH_INFO: `/v3/projects` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:28:19.596507 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb5e384-21db-4ac9-a2d5-d2c6a48b7700 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:28:19.598070 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb5e384-21db-4ac9-a2d5-d2c6a48b7700 None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:28:19.598902 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 467/943] 10.222.0.57 () {56 vars in 1122 bytes} [Fri Jul 1 16:28:19 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:28:19.600052 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41afaa1-3c4b-4819-8276-936b480bdaa2 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:28:19.600190 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: WARNING keystone.server.flask.common [None req-1eb5e384-21db-4ac9-a2d5-d2c6a48b7700 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Jul 01 16:28:19.622085 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.sql.core [None req-1eb5e384-21db-4ac9-a2d5-d2c6a48b7700 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 01 16:28:19.622085 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jul 01 16:28:19.622085 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [parameters: {'id': '972ad5c8e4044db8934a366361387e1a', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jul 01 16:28:19.622085 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80312) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 01 16:28:19.631396 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application [None req-1eb5e384-21db-4ac9-a2d5-d2c6a48b7700 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 972ad5c8e4044db8934a366361387e1a.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 972ad5c8e4044db8934a366361387e1a. Jul 01 16:28:19.631396 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 16:28:19.631396 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 01 16:28:19.631396 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 01 16:28:19.631396 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 01 16:28:19.631396 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 01 16:28:19.631396 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 01 16:28:19.631396 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application result = self._query(query) Jul 01 16:28:19.631396 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 01 16:28:19.631396 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application conn.query(q) Jul 01 16:28:19.631396 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 01 16:28:19.631396 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 01 16:28:19.631396 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 01 16:28:19.632278 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application result.read() Jul 01 16:28:19.632278 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 01 16:28:19.632278 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 01 16:28:19.632278 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 01 16:28:19.632278 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 01 16:28:19.632278 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 01 16:28:19.632278 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 01 16:28:19.632278 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 01 16:28:19.632278 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 01 16:28:19.632278 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 01 16:28:19.632278 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application Jul 01 16:28:19.632278 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 01 16:28:19.632278 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application Jul 01 16:28:19.632278 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 16:28:19.632278 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 01 16:28:19.632278 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 01 16:28:19.632278 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Jul 01 16:28:19.632278 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return project_ref.to_dict() Jul 01 16:28:19.633684 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 01 16:28:19.633684 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application next(self.gen) Jul 01 16:28:19.633684 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 01 16:28:19.633684 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application yield resource Jul 01 16:28:19.633684 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 01 16:28:19.633684 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application next(self.gen) Jul 01 16:28:19.633684 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 01 16:28:19.633684 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application self.session.rollback() Jul 01 16:28:19.633684 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 01 16:28:19.633684 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application self.force_reraise() Jul 01 16:28:19.633684 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 01 16:28:19.633684 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application raise self.value Jul 01 16:28:19.633684 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 01 16:28:19.633684 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 01 16:28:19.633684 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 01 16:28:19.633684 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application session.commit() Jul 01 16:28:19.633684 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 01 16:28:19.635938 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 01 16:28:19.635938 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 01 16:28:19.635938 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application self._prepare_impl() Jul 01 16:28:19.635938 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 01 16:28:19.635938 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application self.session.flush() Jul 01 16:28:19.635938 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 01 16:28:19.635938 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application self._flush(objects) Jul 01 16:28:19.635938 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 01 16:28:19.635938 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 01 16:28:19.635938 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 01 16:28:19.635938 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application compat.raise_( Jul 01 16:28:19.635938 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 01 16:28:19.635938 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application raise exception Jul 01 16:28:19.635938 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 01 16:28:19.635938 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application flush_context.execute() Jul 01 16:28:19.635938 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 01 16:28:19.635938 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application rec.execute(self) Jul 01 16:28:19.642103 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 01 16:28:19.642103 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 01 16:28:19.642103 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 01 16:28:19.642103 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 01 16:28:19.642103 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jul 01 16:28:19.642103 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application c = connection._execute_20( Jul 01 16:28:19.642103 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 01 16:28:19.642103 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 01 16:28:19.642103 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 01 16:28:19.642103 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 01 16:28:19.642103 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 01 16:28:19.642103 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 01 16:28:19.642103 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 01 16:28:19.642103 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 01 16:28:19.642103 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 01 16:28:19.644097 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 01 16:28:19.644097 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 01 16:28:19.644097 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application raise exception Jul 01 16:28:19.644097 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 01 16:28:19.644097 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 01 16:28:19.644097 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 01 16:28:19.644097 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 01 16:28:19.644097 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 01 16:28:19.644097 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application result = self._query(query) Jul 01 16:28:19.644097 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 01 16:28:19.644097 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application conn.query(q) Jul 01 16:28:19.644097 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 01 16:28:19.644097 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 01 16:28:19.644097 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 01 16:28:19.644097 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application result.read() Jul 01 16:28:19.644097 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 01 16:28:19.644097 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 01 16:28:19.647297 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 01 16:28:19.647297 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 01 16:28:19.647297 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 01 16:28:19.647297 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 01 16:28:19.647297 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 01 16:28:19.647297 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 01 16:28:19.647297 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 01 16:28:19.647297 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jul 01 16:28:19.647297 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application [parameters: {'id': '972ad5c8e4044db8934a366361387e1a', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jul 01 16:28:19.647297 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 01 16:28:19.647297 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application Jul 01 16:28:19.647297 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 01 16:28:19.647297 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application Jul 01 16:28:19.647297 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 16:28:19.647297 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Jul 01 16:28:19.655073 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Jul 01 16:28:19.655073 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 01 16:28:19.655073 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 01 16:28:19.655073 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Jul 01 16:28:19.655073 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application Jul 01 16:28:19.655073 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 01 16:28:19.655073 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application Jul 01 16:28:19.655073 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 16:28:19.655073 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 16:28:19.655073 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 16:28:19.655073 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 16:28:19.655073 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 16:28:19.655073 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 16:28:19.655073 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 16:28:19.655073 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 16:28:19.655073 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 16:28:19.655073 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 16:28:19.657254 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 16:28:19.657254 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Jul 01 16:28:19.657254 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Jul 01 16:28:19.657254 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 16:28:19.657254 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 16:28:19.657254 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Jul 01 16:28:19.657254 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application raise exception.Conflict( Jul 01 16:28:19.657254 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 972ad5c8e4044db8934a366361387e1a. Jul 01 16:28:19.657254 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application Jul 01 16:28:19.657254 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 477/944] 10.222.0.57 () {60 vars in 1140 bytes} [Fri Jul 1 16:28:19 2022] POST /identity/v3/projects => generated 262 bytes in 219 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jul 01 16:28:19.657254 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb5e384-21db-4ac9-a2d5-d2c6a48b7700 None admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:28:19.658525 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:28:19.747573 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41afaa1-3c4b-4819-8276-936b480bdaa2 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:28:19.781918 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb5e384-21db-4ac9-a2d5-d2c6a48b7700 None admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:28:19.795497 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4c04fd-23b5-4271-b271-6f4978193192 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:28:19.796479 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4c04fd-23b5-4271-b271-6f4978193192 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:28:19.797224 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4c04fd-23b5-4271-b271-6f4978193192 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:28:19.798164 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4c04fd-23b5-4271-b271-6f4978193192 None admin] PATH_INFO: `/v3/services` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:28:19.799422 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c4c04fd-23b5-4271-b271-6f4978193192 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:28:19.799948 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c4c04fd-23b5-4271-b271-6f4978193192 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:28:19.802368 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c4c04fd-23b5-4271-b271-6f4978193192 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:28:19.810383 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 468/945] 10.222.0.57 () {56 vars in 1122 bytes} [Fri Jul 1 16:28:19 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:28:19.816375 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4c04fd-23b5-4271-b271-6f4978193192 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:28:19.820918 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255d1abf-902d-46d3-9179-847265574f56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:28:19.822436 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255d1abf-902d-46d3-9179-847265574f56 None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:28:19.822436 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255d1abf-902d-46d3-9179-847265574f56 None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:28:19.822436 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255d1abf-902d-46d3-9179-847265574f56 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:28:19.829869 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-255d1abf-902d-46d3-9179-847265574f56 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:28:19.831959 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-255d1abf-902d-46d3-9179-847265574f56 None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:28:19.846596 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application [None req-255d1abf-902d-46d3-9179-847265574f56 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 01 16:28:19.846596 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 16:28:19.846596 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 16:28:19.846596 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 16:28:19.846596 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 16:28:19.846596 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 16:28:19.846596 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 16:28:19.846596 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 16:28:19.846596 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 16:28:19.846596 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 16:28:19.846596 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 16:28:19.846596 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 16:28:19.846596 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 01 16:28:19.846596 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 01 16:28:19.846596 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 01 16:28:19.847567 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 01 16:28:19.847567 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 16:28:19.847567 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 16:28:19.847567 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 16:28:19.847567 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 16:28:19.847567 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 16:28:19.847567 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 16:28:19.847567 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 16:28:19.847567 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application with Lock( Jul 01 16:28:19.847567 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 16:28:19.847567 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self._enter() Jul 01 16:28:19.847567 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 16:28:19.847567 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 16:28:19.847567 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 16:28:19.847567 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.creator() Jul 01 16:28:19.847567 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 16:28:19.847567 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application created_value = creator( Jul 01 16:28:19.849069 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 01 16:28:19.849069 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 01 16:28:19.849069 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 01 16:28:19.849069 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 01 16:28:19.849069 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 01 16:28:19.849069 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 01 16:28:19.849069 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 01 16:28:19.849069 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application Jul 01 16:28:19.853765 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 478/946] 10.222.0.57 () {56 vars in 1111 bytes} [Fri Jul 1 16:28:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 01 16:28:19.854871 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255d1abf-902d-46d3-9179-847265574f56 None admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:28:19.902228 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:28:19.968138 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4c04fd-23b5-4271-b271-6f4978193192 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:28:19.976222 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255d1abf-902d-46d3-9179-847265574f56 None admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:28:19.987314 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:28:20.012955 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b48d545-661a-401d-a2a9-ed2e6d5e4166 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:28:20.013760 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b48d545-661a-401d-a2a9-ed2e6d5e4166 None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:28:20.014116 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b48d545-661a-401d-a2a9-ed2e6d5e4166 None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:28:20.014569 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b48d545-661a-401d-a2a9-ed2e6d5e4166 None admin] PATH_INFO: `/v3/projects` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:28:20.015561 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b48d545-661a-401d-a2a9-ed2e6d5e4166 None admin] RBAC: Adding query filter params (name=service) {{(pid=80312) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:28:20.015949 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b48d545-661a-401d-a2a9-ed2e6d5e4166 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:28:20.016742 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178fd29f-136a-410f-bf4d-7df3c0e111a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:28:20.017526 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b48d545-661a-401d-a2a9-ed2e6d5e4166 None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:28:20.017813 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178fd29f-136a-410f-bf4d-7df3c0e111a2 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:28:20.018329 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178fd29f-136a-410f-bf4d-7df3c0e111a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:28:20.018825 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178fd29f-136a-410f-bf4d-7df3c0e111a2 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:28:20.019861 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-178fd29f-136a-410f-bf4d-7df3c0e111a2 None admin] RBAC: Adding query filter params (interface=public, service_id=869b98028e3547e3b6563b17f76e37cb, region_id=RegionOne) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:28:20.020373 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-178fd29f-136a-410f-bf4d-7df3c0e111a2 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:28:20.021913 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-178fd29f-136a-410f-bf4d-7df3c0e111a2 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:28:20.028133 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 469/947] 10.222.0.57 () {56 vars in 1251 bytes} [Fri Jul 1 16:28:19 2022] GET /identity/v3/endpoints?service_id=869b98028e3547e3b6563b17f76e37cb&interface=public®ion_id=RegionOne => generated 194 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 01 16:28:20.038400 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178fd29f-136a-410f-bf4d-7df3c0e111a2 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:28:20.046569 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 479/948] 10.222.0.57 () {56 vars in 1112 bytes} [Fri Jul 1 16:28:19 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:28:20.158652 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178fd29f-136a-410f-bf4d-7df3c0e111a2 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:28:20.199942 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ac357e-eb1d-49f4-a6e4-90f8f85cc22d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:28:20.200723 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ac357e-eb1d-49f4-a6e4-90f8f85cc22d None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:28:20.202007 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ac357e-eb1d-49f4-a6e4-90f8f85cc22d None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:28:20.202007 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ac357e-eb1d-49f4-a6e4-90f8f85cc22d None admin] PATH_INFO: `/v3/services` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:28:20.202007 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ac357e-eb1d-49f4-a6e4-90f8f85cc22d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:28:20.204570 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ac357e-eb1d-49f4-a6e4-90f8f85cc22d None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:28:20.213018 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 470/949] 10.222.0.57 () {56 vars in 1087 bytes} [Fri Jul 1 16:28:20 2022] GET /identity/v3/services => generated 1497 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 16:28:20.516115 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971bd942-60b5-454f-aec7-32d36963122b None None] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:28:20.516609 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971bd942-60b5-454f-aec7-32d36963122b None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:28:20.516609 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971bd942-60b5-454f-aec7-32d36963122b None None] PATH_INFO: `/` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:28:20.518228 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 480/950] 10.222.0.57 () {54 vars in 926 bytes} [Fri Jul 1 16:28:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 16:28:20.524407 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d538670d-5363-4d7d-8cbb-a63c3cb0f500 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:28:20.524709 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d538670d-5363-4d7d-8cbb-a63c3cb0f500 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:28:20.524977 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d538670d-5363-4d7d-8cbb-a63c3cb0f500 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:28:20.542431 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-d538670d-5363-4d7d-8cbb-a63c3cb0f500 None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:28:20.543617 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-d538670d-5363-4d7d-8cbb-a63c3cb0f500 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:28:20.583938 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:28:20.622389 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 471/951] 10.222.0.57 () {58 vars in 1023 bytes} [Fri Jul 1 16:28:20 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 01 16:28:20.648667 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971bd942-60b5-454f-aec7-32d36963122b None None] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:28:20.753776 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=100 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:28:20.759182 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971bd942-60b5-454f-aec7-32d36963122b None None] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:28:20.795238 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1002653-34dc-4e65-88de-70f6ac338cd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:28:20.796045 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1002653-34dc-4e65-88de-70f6ac338cd4 None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:28:20.796328 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1002653-34dc-4e65-88de-70f6ac338cd4 None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:28:20.796608 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1002653-34dc-4e65-88de-70f6ac338cd4 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:28:20.797272 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1002653-34dc-4e65-88de-70f6ac338cd4 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:28:20.798726 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1002653-34dc-4e65-88de-70f6ac338cd4 None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:28:20.811372 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application [None req-e1002653-34dc-4e65-88de-70f6ac338cd4 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 01 16:28:20.811372 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 16:28:20.811372 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 01 16:28:20.811372 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 01 16:28:20.811372 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 01 16:28:20.811372 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 01 16:28:20.811372 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 01 16:28:20.811372 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 01 16:28:20.811372 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 01 16:28:20.811372 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application Jul 01 16:28:20.811372 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 01 16:28:20.811372 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application Jul 01 16:28:20.811372 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 16:28:20.811372 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 16:28:20.811372 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 16:28:20.811372 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 16:28:20.811372 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 16:28:20.812383 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 16:28:20.812383 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 16:28:20.812383 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 16:28:20.812383 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 16:28:20.812383 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 16:28:20.812383 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 16:28:20.812383 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 01 16:28:20.812383 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 01 16:28:20.812383 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 01 16:28:20.812383 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 01 16:28:20.812383 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 16:28:20.812383 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 16:28:20.812383 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 16:28:20.812383 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 16:28:20.812383 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 16:28:20.812383 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 16:28:20.813245 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 16:28:20.813245 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application with Lock( Jul 01 16:28:20.813245 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 16:28:20.813245 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self._enter() Jul 01 16:28:20.813245 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 16:28:20.813245 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 16:28:20.813245 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 16:28:20.813245 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.creator() Jul 01 16:28:20.813245 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 16:28:20.813245 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application created_value = creator( Jul 01 16:28:20.813245 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 01 16:28:20.813245 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 01 16:28:20.813245 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 01 16:28:20.813245 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application Jul 01 16:28:20.813245 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 481/952] 10.222.0.57 () {56 vars in 1111 bytes} [Fri Jul 1 16:28:20 2022] GET /identity/v3/services/compute => generated 88 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 16:28:20.819400 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d538670d-5363-4d7d-8cbb-a63c3cb0f500 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:28:20.930151 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d538670d-5363-4d7d-8cbb-a63c3cb0f500 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:28:20.967059 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71cf0094-805d-4b58-85fa-51b56c9e2626 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:28:20.967742 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71cf0094-805d-4b58-85fa-51b56c9e2626 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:28:20.968034 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71cf0094-805d-4b58-85fa-51b56c9e2626 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:28:20.968318 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71cf0094-805d-4b58-85fa-51b56c9e2626 None admin] PATH_INFO: `/v3/services` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:28:20.969035 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71cf0094-805d-4b58-85fa-51b56c9e2626 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:28:20.969348 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71cf0094-805d-4b58-85fa-51b56c9e2626 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:28:20.970829 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71cf0094-805d-4b58-85fa-51b56c9e2626 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:28:20.977072 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 472/953] 10.222.0.57 () {56 vars in 1112 bytes} [Fri Jul 1 16:28:20 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:28:20.982497 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1002653-34dc-4e65-88de-70f6ac338cd4 None admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:28:21.091962 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1002653-34dc-4e65-88de-70f6ac338cd4 None admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:28:21.126764 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8238a766-a394-47b6-b820-be788de8e8e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:28:21.127623 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8238a766-a394-47b6-b820-be788de8e8e1 None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:28:21.127912 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8238a766-a394-47b6-b820-be788de8e8e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:28:21.128193 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8238a766-a394-47b6-b820-be788de8e8e1 None admin] PATH_INFO: `/v3/services` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:28:21.128908 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8238a766-a394-47b6-b820-be788de8e8e1 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80312) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:28:21.129780 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8238a766-a394-47b6-b820-be788de8e8e1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:28:21.131286 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8238a766-a394-47b6-b820-be788de8e8e1 None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:28:21.136420 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 482/954] 10.222.0.57 () {56 vars in 1112 bytes} [Fri Jul 1 16:28:20 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:28:21.395242 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d93d98-5449-403d-b812-870bcf11f535 None None] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:28:21.395743 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d93d98-5449-403d-b812-870bcf11f535 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:28:21.396014 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d93d98-5449-403d-b812-870bcf11f535 None None] PATH_INFO: `/` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:28:21.397796 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 473/955] 10.222.0.57 () {54 vars in 926 bytes} [Fri Jul 1 16:28:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 16:28:21.403267 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f07881a-0db3-4bd9-a948-5c8ddbd3e3f7 None None] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:28:21.403679 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f07881a-0db3-4bd9-a948-5c8ddbd3e3f7 None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:28:21.404065 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f07881a-0db3-4bd9-a948-5c8ddbd3e3f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:28:21.422571 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.auth.core [None req-1f07881a-0db3-4bd9-a948-5c8ddbd3e3f7 None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80312) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:28:21.424084 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-1f07881a-0db3-4bd9-a948-5c8ddbd3e3f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:28:21.446479 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=100 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:28:21.498135 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 483/956] 10.222.0.57 () {58 vars in 1023 bytes} [Fri Jul 1 16:28:21 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 01 16:28:21.524733 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d93d98-5449-403d-b812-870bcf11f535 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:28:21.563715 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:28:21.649962 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d93d98-5449-403d-b812-870bcf11f535 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:28:21.689080 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69536dbf-2b87-4da4-a689-671ee24a0647 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:28:21.689752 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69536dbf-2b87-4da4-a689-671ee24a0647 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:28:21.690043 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69536dbf-2b87-4da4-a689-671ee24a0647 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:28:21.690452 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69536dbf-2b87-4da4-a689-671ee24a0647 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:28:21.703143 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69536dbf-2b87-4da4-a689-671ee24a0647 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 01 16:28:21.711744 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application [None req-69536dbf-2b87-4da4-a689-671ee24a0647 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 01 16:28:21.711744 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 16:28:21.711744 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 16:28:21.711744 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 16:28:21.711744 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 16:28:21.711744 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 16:28:21.711744 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 16:28:21.711744 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 16:28:21.711744 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 16:28:21.711744 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 16:28:21.711744 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 16:28:21.711744 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 16:28:21.711744 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 01 16:28:21.711744 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 01 16:28:21.711744 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 01 16:28:21.711744 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 01 16:28:21.713736 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 01 16:28:21.713736 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 01 16:28:21.713736 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 01 16:28:21.713736 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 01 16:28:21.713736 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 16:28:21.713736 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 16:28:21.713736 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 16:28:21.713736 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 16:28:21.713736 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 16:28:21.713736 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 16:28:21.713736 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 16:28:21.713736 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application with Lock( Jul 01 16:28:21.713736 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 16:28:21.713736 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._enter() Jul 01 16:28:21.713736 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 16:28:21.713736 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 16:28:21.714939 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 16:28:21.714939 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.creator() Jul 01 16:28:21.714939 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 16:28:21.714939 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application created_value = creator( Jul 01 16:28:21.714939 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 01 16:28:21.714939 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 01 16:28:21.714939 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 01 16:28:21.714939 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 01 16:28:21.714939 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 01 16:28:21.714939 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 01 16:28:21.714939 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 01 16:28:21.714939 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Jul 01 16:28:21.714939 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 474/957] 10.222.0.57 () {56 vars in 1120 bytes} [Fri Jul 1 16:28:21 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 16:28:21.719351 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f07881a-0db3-4bd9-a948-5c8ddbd3e3f7 None None] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:28:21.836436 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f07881a-0db3-4bd9-a948-5c8ddbd3e3f7 None None] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:28:21.876442 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c625d0-6b5c-49d0-be87-057735199285 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:28:21.877116 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c625d0-6b5c-49d0-be87-057735199285 None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:28:21.877400 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c625d0-6b5c-49d0-be87-057735199285 None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:28:21.877751 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c625d0-6b5c-49d0-be87-057735199285 None admin] PATH_INFO: `/v3/roles` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:28:21.878535 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c625d0-6b5c-49d0-be87-057735199285 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80312) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:28:21.878845 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c625d0-6b5c-49d0-be87-057735199285 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:28:21.880609 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c625d0-6b5c-49d0-be87-057735199285 None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:28:21.891338 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 484/958] 10.222.0.57 () {56 vars in 1115 bytes} [Fri Jul 1 16:28:21 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 01 16:28:21.900107 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69536dbf-2b87-4da4-a689-671ee24a0647 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:28:22.033525 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:28:22.043912 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69536dbf-2b87-4da4-a689-671ee24a0647 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:28:22.085369 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8911a0-dae4-4ffc-8979-1313c9eb7d69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:28:22.086566 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8911a0-dae4-4ffc-8979-1313c9eb7d69 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:28:22.087048 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8911a0-dae4-4ffc-8979-1313c9eb7d69 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:28:22.087680 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8911a0-dae4-4ffc-8979-1313c9eb7d69 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:28:22.103526 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b8911a0-dae4-4ffc-8979-1313c9eb7d69 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:28:22.105638 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b8911a0-dae4-4ffc-8979-1313c9eb7d69 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:28:22.126458 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application [None req-8b8911a0-dae4-4ffc-8979-1313c9eb7d69 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 16:28:22.126458 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 16:28:22.126458 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 16:28:22.126458 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 16:28:22.126458 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 16:28:22.126458 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 16:28:22.126458 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 16:28:22.126458 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 16:28:22.126458 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 16:28:22.126458 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 16:28:22.126458 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 16:28:22.126458 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 16:28:22.126458 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 01 16:28:22.126458 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 01 16:28:22.126458 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 01 16:28:22.127400 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 01 16:28:22.127400 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 16:28:22.127400 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 16:28:22.127400 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 16:28:22.127400 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 16:28:22.127400 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 16:28:22.127400 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 16:28:22.127400 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 16:28:22.127400 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application with Lock( Jul 01 16:28:22.127400 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 16:28:22.127400 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._enter() Jul 01 16:28:22.127400 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 16:28:22.127400 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 16:28:22.127400 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 16:28:22.127400 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.creator() Jul 01 16:28:22.127400 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 16:28:22.127400 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application created_value = creator( Jul 01 16:28:22.128295 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 01 16:28:22.128295 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 01 16:28:22.128295 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 16:28:22.128295 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Jul 01 16:28:22.131119 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 475/959] 10.222.0.57 () {56 vars in 1108 bytes} [Fri Jul 1 16:28:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 16:28:22.137922 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c625d0-6b5c-49d0-be87-057735199285 None admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:28:22.162511 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=100 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:28:22.271892 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c625d0-6b5c-49d0-be87-057735199285 None admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:28:22.282340 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4498501d-d069-4eb6-80e6-5ea2930f2df5 None None] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:28:22.282668 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4498501d-d069-4eb6-80e6-5ea2930f2df5 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:28:22.282933 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4498501d-d069-4eb6-80e6-5ea2930f2df5 None None] PATH_INFO: `/` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:28:22.286241 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 476/960] 10.222.0.57 () {54 vars in 926 bytes} [Fri Jul 1 16:28:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 16:28:22.292038 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7eb937-bcff-4274-ac5a-453c61485378 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:28:22.293074 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7eb937-bcff-4274-ac5a-453c61485378 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:28:22.293677 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7eb937-bcff-4274-ac5a-453c61485378 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:28:22.316186 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95bb70ea-b237-44f9-966d-bd61d08424ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:28:22.317672 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bb70ea-b237-44f9-966d-bd61d08424ae None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:28:22.318176 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bb70ea-b237-44f9-966d-bd61d08424ae None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:28:22.318538 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bb70ea-b237-44f9-966d-bd61d08424ae None admin] PATH_INFO: `/v3/domains` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:28:22.319319 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95bb70ea-b237-44f9-966d-bd61d08424ae None admin] RBAC: Adding query filter params (name=Default) {{(pid=80312) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:28:22.319482 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-aa7eb937-bcff-4274-ac5a-453c61485378 None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:28:22.319639 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95bb70ea-b237-44f9-966d-bd61d08424ae None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:28:22.320563 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-aa7eb937-bcff-4274-ac5a-453c61485378 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:28:22.321013 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95bb70ea-b237-44f9-966d-bd61d08424ae None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:28:22.341617 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 485/961] 10.222.0.57 () {56 vars in 1109 bytes} [Fri Jul 1 16:28:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:28:22.347560 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95bb70ea-b237-44f9-966d-bd61d08424ae None admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:28:22.410199 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 477/962] 10.222.0.57 () {58 vars in 1023 bytes} [Fri Jul 1 16:28:22 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 01 16:28:22.430741 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7eb937-bcff-4274-ac5a-453c61485378 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:28:22.454537 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=100 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:28:22.486145 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95bb70ea-b237-44f9-966d-bd61d08424ae None admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:28:22.526809 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7a4656-2cc0-41cf-aee8-52150f5d17a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:28:22.527615 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7a4656-2cc0-41cf-aee8-52150f5d17a3 None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:28:22.527897 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7a4656-2cc0-41cf-aee8-52150f5d17a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:28:22.528167 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7a4656-2cc0-41cf-aee8-52150f5d17a3 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:28:22.530006 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:28:22.538475 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b7a4656-2cc0-41cf-aee8-52150f5d17a3 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:28:22.539918 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b7a4656-2cc0-41cf-aee8-52150f5d17a3 None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:28:22.560261 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application [None req-2b7a4656-2cc0-41cf-aee8-52150f5d17a3 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 01 16:28:22.560261 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 16:28:22.560261 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 16:28:22.560261 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 16:28:22.560261 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 16:28:22.560261 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 16:28:22.560261 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 16:28:22.560261 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 16:28:22.560261 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 16:28:22.560261 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 16:28:22.560261 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 16:28:22.560261 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 16:28:22.560261 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 01 16:28:22.560261 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 01 16:28:22.560261 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 01 16:28:22.560261 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 01 16:28:22.561609 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 16:28:22.561609 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 16:28:22.561609 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 01 16:28:22.561609 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 01 16:28:22.561609 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 01 16:28:22.561609 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 01 16:28:22.561609 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 16:28:22.561609 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 16:28:22.561609 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 16:28:22.561609 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 16:28:22.561609 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 16:28:22.561609 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application with Lock( Jul 01 16:28:22.561609 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 16:28:22.561609 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self._enter() Jul 01 16:28:22.561609 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 16:28:22.561609 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 16:28:22.561609 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 16:28:22.561609 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.creator() Jul 01 16:28:22.563036 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 16:28:22.563036 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application created_value = creator( Jul 01 16:28:22.563036 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 01 16:28:22.563036 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 01 16:28:22.563036 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 01 16:28:22.563036 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 01 16:28:22.563036 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 01 16:28:22.563036 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 01 16:28:22.563036 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 01 16:28:22.563036 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application Jul 01 16:28:22.563638 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7eb937-bcff-4274-ac5a-453c61485378 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:28:22.567358 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 486/963] 10.222.0.57 () {56 vars in 1117 bytes} [Fri Jul 1 16:28:22 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 16:28:22.568848 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15aa07f2-bd79-49f4-8334-95ccf377ec15 None None] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:28:22.569479 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15aa07f2-bd79-49f4-8334-95ccf377ec15 None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:28:22.569479 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15aa07f2-bd79-49f4-8334-95ccf377ec15 None None] PATH_INFO: `/` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:28:22.574481 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 487/964] 10.222.0.57 () {54 vars in 926 bytes} [Fri Jul 1 16:28:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 16:28:22.574481 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15aa07f2-bd79-49f4-8334-95ccf377ec15 None None] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:28:22.614771 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701819e3-c1bb-404c-bc34-e5c22462b9f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:28:22.616274 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701819e3-c1bb-404c-bc34-e5c22462b9f8 None admin] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:28:22.616449 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701819e3-c1bb-404c-bc34-e5c22462b9f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:28:22.616797 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701819e3-c1bb-404c-bc34-e5c22462b9f8 None admin] PATH_INFO: `/v3/users` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:28:22.617576 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701819e3-c1bb-404c-bc34-e5c22462b9f8 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:28:22.619313 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701819e3-c1bb-404c-bc34-e5c22462b9f8 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:28:22.621127 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: WARNING keystone.server.flask.common [None req-701819e3-c1bb-404c-bc34-e5c22462b9f8 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Jul 01 16:28:22.638763 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.sql.core [None req-701819e3-c1bb-404c-bc34-e5c22462b9f8 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 01 16:28:22.638763 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jul 01 16:28:22.638763 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [parameters: {'user_id': '68b7b81a022140b3b338509e3fbeec35', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jul 01 16:28:22.638763 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80311) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 01 16:28:22.662873 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application [None req-701819e3-c1bb-404c-bc34-e5c22462b9f8 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jul 01 16:28:22.662873 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 16:28:22.662873 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 01 16:28:22.662873 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 01 16:28:22.662873 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 01 16:28:22.662873 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 01 16:28:22.662873 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 01 16:28:22.662873 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application result = self._query(query) Jul 01 16:28:22.662873 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 01 16:28:22.662873 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application conn.query(q) Jul 01 16:28:22.662873 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 01 16:28:22.662873 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 01 16:28:22.662873 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 01 16:28:22.662873 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application result.read() Jul 01 16:28:22.662873 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 01 16:28:22.663887 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 01 16:28:22.663887 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 01 16:28:22.663887 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 01 16:28:22.663887 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 01 16:28:22.663887 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 01 16:28:22.663887 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 01 16:28:22.663887 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 01 16:28:22.663887 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 01 16:28:22.663887 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Jul 01 16:28:22.663887 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 01 16:28:22.663887 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Jul 01 16:28:22.663887 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 16:28:22.663887 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 01 16:28:22.663887 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 01 16:28:22.663887 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Jul 01 16:28:22.663887 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Jul 01 16:28:22.663887 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 01 16:28:22.663887 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application next(self.gen) Jul 01 16:28:22.664881 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 01 16:28:22.664881 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application yield resource Jul 01 16:28:22.664881 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 01 16:28:22.664881 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application next(self.gen) Jul 01 16:28:22.664881 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 01 16:28:22.664881 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application self.session.rollback() Jul 01 16:28:22.664881 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 01 16:28:22.664881 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application self.force_reraise() Jul 01 16:28:22.664881 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 01 16:28:22.664881 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise self.value Jul 01 16:28:22.664881 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 01 16:28:22.664881 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 01 16:28:22.664881 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 01 16:28:22.664881 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application session.commit() Jul 01 16:28:22.664881 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 01 16:28:22.664881 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 01 16:28:22.664881 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 01 16:28:22.665775 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application self._prepare_impl() Jul 01 16:28:22.665775 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 01 16:28:22.665775 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application self.session.flush() Jul 01 16:28:22.665775 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 01 16:28:22.665775 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application self._flush(objects) Jul 01 16:28:22.665775 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 01 16:28:22.665775 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 01 16:28:22.665775 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 01 16:28:22.665775 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application compat.raise_( Jul 01 16:28:22.665775 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 01 16:28:22.665775 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception Jul 01 16:28:22.665775 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 01 16:28:22.665775 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application flush_context.execute() Jul 01 16:28:22.665775 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 01 16:28:22.665775 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application rec.execute(self) Jul 01 16:28:22.665775 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 01 16:28:22.665775 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 01 16:28:22.667529 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 01 16:28:22.667529 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 01 16:28:22.667529 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Jul 01 16:28:22.667529 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application result = connection._execute_20( Jul 01 16:28:22.667529 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 01 16:28:22.667529 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 01 16:28:22.667529 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 01 16:28:22.667529 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 01 16:28:22.667529 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 01 16:28:22.667529 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 01 16:28:22.667529 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 01 16:28:22.667529 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 01 16:28:22.667529 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 01 16:28:22.667529 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 01 16:28:22.667529 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 01 16:28:22.667529 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception Jul 01 16:28:22.669804 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 01 16:28:22.669804 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 01 16:28:22.669804 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 01 16:28:22.669804 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 01 16:28:22.669804 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 01 16:28:22.669804 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application result = self._query(query) Jul 01 16:28:22.669804 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 01 16:28:22.669804 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application conn.query(q) Jul 01 16:28:22.669804 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 01 16:28:22.669804 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 01 16:28:22.669804 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 01 16:28:22.669804 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application result.read() Jul 01 16:28:22.669804 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 01 16:28:22.669804 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 01 16:28:22.669804 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 01 16:28:22.669804 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 01 16:28:22.672797 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 01 16:28:22.672797 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 01 16:28:22.672797 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 01 16:28:22.672797 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 01 16:28:22.672797 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 01 16:28:22.672797 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jul 01 16:28:22.672797 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application [parameters: {'user_id': '68b7b81a022140b3b338509e3fbeec35', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jul 01 16:28:22.672797 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 01 16:28:22.672797 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Jul 01 16:28:22.672797 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 01 16:28:22.672797 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Jul 01 16:28:22.672797 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 16:28:22.672797 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 16:28:22.672797 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 16:28:22.672797 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 16:28:22.674946 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 16:28:22.674946 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 16:28:22.674946 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 16:28:22.674946 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 16:28:22.674946 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 16:28:22.674946 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 16:28:22.674946 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 16:28:22.674946 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Jul 01 16:28:22.674946 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Jul 01 16:28:22.674946 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 16:28:22.674946 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 16:28:22.674946 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 01 16:28:22.674946 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 01 16:28:22.674946 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 01 16:28:22.674946 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 01 16:28:22.674946 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Jul 01 16:28:22.674946 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Jul 01 16:28:22.677629 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Jul 01 16:28:22.677629 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Jul 01 16:28:22.677629 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 01 16:28:22.677629 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 01 16:28:22.677629 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jul 01 16:28:22.677629 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Jul 01 16:28:22.677629 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 478/965] 10.222.0.57 () {60 vars in 1131 bytes} [Fri Jul 1 16:28:22 2022] POST /identity/v3/users => generated 160 bytes in 239 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Jul 01 16:28:22.677629 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f611ca42-8486-4f71-925b-1b9248c58815 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:28:22.677629 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f611ca42-8486-4f71-925b-1b9248c58815 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:28:22.677629 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f611ca42-8486-4f71-925b-1b9248c58815 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:28:22.692159 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-f611ca42-8486-4f71-925b-1b9248c58815 None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:28:22.693734 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-f611ca42-8486-4f71-925b-1b9248c58815 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:28:22.727114 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15aa07f2-bd79-49f4-8334-95ccf377ec15 None None] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:28:22.769520 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc30ecce-4686-49f2-b31e-ace429d799cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:28:22.770210 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc30ecce-4686-49f2-b31e-ace429d799cc None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:28:22.770561 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc30ecce-4686-49f2-b31e-ace429d799cc None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:28:22.770913 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc30ecce-4686-49f2-b31e-ace429d799cc None admin] PATH_INFO: `/v3/users` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:28:22.771965 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc30ecce-4686-49f2-b31e-ace429d799cc None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80312) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:28:22.772286 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc30ecce-4686-49f2-b31e-ace429d799cc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:28:22.774009 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc30ecce-4686-49f2-b31e-ace429d799cc None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:28:22.786564 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=100 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:28:22.791340 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 479/966] 10.222.0.57 () {58 vars in 1023 bytes} [Fri Jul 1 16:28:22 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 01 16:28:22.792182 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f611ca42-8486-4f71-925b-1b9248c58815 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:28:22.818412 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 488/967] 10.222.0.57 () {56 vars in 1149 bytes} [Fri Jul 1 16:28:22 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:28:22.821830 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc30ecce-4686-49f2-b31e-ace429d799cc None admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:28:22.898388 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:28:22.942400 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc30ecce-4686-49f2-b31e-ace429d799cc None admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:28:22.945743 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f611ca42-8486-4f71-925b-1b9248c58815 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:28:22.979420 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81a3153-18e5-4007-845c-6c4195e74ce9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:28:22.980231 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81a3153-18e5-4007-845c-6c4195e74ce9 None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:28:22.980520 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81a3153-18e5-4007-845c-6c4195e74ce9 None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:28:22.980805 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81a3153-18e5-4007-845c-6c4195e74ce9 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:28:22.981475 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f81a3153-18e5-4007-845c-6c4195e74ce9 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:28:22.982914 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f81a3153-18e5-4007-845c-6c4195e74ce9 None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:28:22.983115 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e9e21a-949b-419e-9071-aaf5eaa741dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:28:22.983906 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e9e21a-949b-419e-9071-aaf5eaa741dc None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:28:22.984192 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e9e21a-949b-419e-9071-aaf5eaa741dc None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:28:22.984468 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e9e21a-949b-419e-9071-aaf5eaa741dc None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:28:22.992336 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e9e21a-949b-419e-9071-aaf5eaa741dc None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:28:22.993974 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e9e21a-949b-419e-9071-aaf5eaa741dc None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:28:22.996450 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application [None req-f81a3153-18e5-4007-845c-6c4195e74ce9 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 01 16:28:22.996450 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 16:28:22.996450 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 01 16:28:22.996450 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 01 16:28:22.996450 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 01 16:28:22.996450 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 01 16:28:22.996450 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 01 16:28:22.996450 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 01 16:28:22.996450 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 01 16:28:22.996450 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application Jul 01 16:28:22.996450 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 01 16:28:22.996450 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application Jul 01 16:28:22.996450 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 16:28:22.996450 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 16:28:22.996450 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 16:28:22.996450 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 16:28:22.996450 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 16:28:22.997578 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 16:28:22.997578 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 16:28:22.997578 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 16:28:22.997578 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 16:28:22.997578 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 16:28:22.997578 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 16:28:22.997578 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 01 16:28:22.997578 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 01 16:28:22.997578 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 01 16:28:22.997578 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 01 16:28:22.997578 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 16:28:22.997578 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 16:28:22.997578 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 16:28:22.997578 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 16:28:22.997578 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 16:28:22.997578 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 16:28:22.998694 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 16:28:22.998694 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application with Lock( Jul 01 16:28:22.998694 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 16:28:22.998694 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self._enter() Jul 01 16:28:22.998694 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 16:28:22.998694 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 16:28:22.998694 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 16:28:22.998694 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.creator() Jul 01 16:28:22.998694 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 16:28:22.998694 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application created_value = creator( Jul 01 16:28:22.998694 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 01 16:28:22.998694 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 01 16:28:22.998694 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 01 16:28:22.998694 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application Jul 01 16:28:22.998694 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 489/968] 10.222.0.57 () {56 vars in 1126 bytes} [Fri Jul 1 16:28:22 2022] GET /identity/v3/services/object-store => generated 93 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 01 16:28:23.003283 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81a3153-18e5-4007-845c-6c4195e74ce9 None admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:28:23.023259 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application [None req-98e9e21a-949b-419e-9071-aaf5eaa741dc None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 01 16:28:23.023259 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 16:28:23.023259 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 16:28:23.023259 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 16:28:23.023259 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 16:28:23.023259 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 16:28:23.023259 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 16:28:23.023259 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 16:28:23.023259 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 16:28:23.023259 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 16:28:23.023259 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 16:28:23.023259 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 16:28:23.023259 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 01 16:28:23.023259 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 01 16:28:23.023259 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 01 16:28:23.023259 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 01 16:28:23.024322 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 16:28:23.024322 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 16:28:23.024322 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 01 16:28:23.024322 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 01 16:28:23.024322 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 01 16:28:23.024322 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 01 16:28:23.024322 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 16:28:23.024322 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 16:28:23.024322 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 16:28:23.024322 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 16:28:23.024322 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 16:28:23.024322 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application with Lock( Jul 01 16:28:23.024322 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 16:28:23.024322 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._enter() Jul 01 16:28:23.024322 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 16:28:23.024322 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 16:28:23.024322 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 16:28:23.024322 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.creator() Jul 01 16:28:23.025346 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 16:28:23.025346 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application created_value = creator( Jul 01 16:28:23.025346 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 01 16:28:23.025346 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 01 16:28:23.025346 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 01 16:28:23.025346 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 01 16:28:23.025346 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 01 16:28:23.025346 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 01 16:28:23.025346 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 01 16:28:23.025346 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Jul 01 16:28:23.025346 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 480/969] 10.222.0.57 () {56 vars in 1099 bytes} [Fri Jul 1 16:28:22 2022] GET /identity/v3/users/cinder => generated 84 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 16:28:23.025346 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e9e21a-949b-419e-9071-aaf5eaa741dc None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:28:23.076605 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=100 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:28:23.126120 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81a3153-18e5-4007-845c-6c4195e74ce9 None admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:28:23.154084 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e9e21a-949b-419e-9071-aaf5eaa741dc None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:28:23.167085 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb26e7b8-0b79-4700-a5ec-fbcf74a1a6fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:28:23.167741 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb26e7b8-0b79-4700-a5ec-fbcf74a1a6fa None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:28:23.168025 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb26e7b8-0b79-4700-a5ec-fbcf74a1a6fa None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:28:23.168298 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb26e7b8-0b79-4700-a5ec-fbcf74a1a6fa None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:28:23.189722 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:28:23.199612 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682f48b2-1ab9-4c4d-a878-d544b0e831b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:28:23.200235 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682f48b2-1ab9-4c4d-a878-d544b0e831b0 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:28:23.200506 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682f48b2-1ab9-4c4d-a878-d544b0e831b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:28:23.200776 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682f48b2-1ab9-4c4d-a878-d544b0e831b0 None admin] PATH_INFO: `/v3/services` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:28:23.201485 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-682f48b2-1ab9-4c4d-a878-d544b0e831b0 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:28:23.201791 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-682f48b2-1ab9-4c4d-a878-d544b0e831b0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:28:23.203588 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-682f48b2-1ab9-4c4d-a878-d544b0e831b0 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:28:23.211852 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 481/970] 10.222.0.57 () {56 vars in 1122 bytes} [Fri Jul 1 16:28:23 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:28:23.212299 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682f48b2-1ab9-4c4d-a878-d544b0e831b0 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:28:23.259568 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb26e7b8-0b79-4700-a5ec-fbcf74a1a6fa None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:28:23.261064 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb26e7b8-0b79-4700-a5ec-fbcf74a1a6fa None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:28:23.325189 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=100 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:28:23.327436 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682f48b2-1ab9-4c4d-a878-d544b0e831b0 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:28:23.365542 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcffeda7-0099-4131-9bed-7fc0570bff88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:28:23.366215 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcffeda7-0099-4131-9bed-7fc0570bff88 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:28:23.366521 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcffeda7-0099-4131-9bed-7fc0570bff88 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:28:23.366804 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcffeda7-0099-4131-9bed-7fc0570bff88 None admin] PATH_INFO: `/v3/users` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:28:23.367606 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcffeda7-0099-4131-9bed-7fc0570bff88 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:28:23.367910 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcffeda7-0099-4131-9bed-7fc0570bff88 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:28:23.369331 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcffeda7-0099-4131-9bed-7fc0570bff88 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:28:23.370185 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 490/971] 10.222.0.57 () {58 vars in 1282 bytes} [Fri Jul 1 16:28:22 2022] GET /identity/v3/auth/tokens => generated 1464 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 01 16:28:23.371064 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb26e7b8-0b79-4700-a5ec-fbcf74a1a6fa None admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:28:23.409496 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 482/972] 10.222.0.57 () {56 vars in 1101 bytes} [Fri Jul 1 16:28:23 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:28:23.411263 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f94a97-b873-4cd4-b5bf-e4f052e38456 None None] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:28:23.411832 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f94a97-b873-4cd4-b5bf-e4f052e38456 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:28:23.412083 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f94a97-b873-4cd4-b5bf-e4f052e38456 None None] PATH_INFO: `/` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:28:23.413993 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 483/973] 10.222.0.57 () {54 vars in 926 bytes} [Fri Jul 1 16:28:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 16:28:23.414650 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f94a97-b873-4cd4-b5bf-e4f052e38456 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:28:23.485184 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:28:23.498519 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb26e7b8-0b79-4700-a5ec-fbcf74a1a6fa None admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:28:23.541274 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295e98c4-8c5e-4b79-85b1-fdc996cc58fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:28:23.542277 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295e98c4-8c5e-4b79-85b1-fdc996cc58fb None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:28:23.542810 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295e98c4-8c5e-4b79-85b1-fdc996cc58fb None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:28:23.543279 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295e98c4-8c5e-4b79-85b1-fdc996cc58fb None admin] PATH_INFO: `/v3/services` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:28:23.544105 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-295e98c4-8c5e-4b79-85b1-fdc996cc58fb None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80312) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:28:23.544411 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-295e98c4-8c5e-4b79-85b1-fdc996cc58fb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:28:23.545816 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-295e98c4-8c5e-4b79-85b1-fdc996cc58fb None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:28:23.553220 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 491/974] 10.222.0.57 () {56 vars in 1122 bytes} [Fri Jul 1 16:28:23 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:28:23.554644 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a587b1b9-29ca-4623-b6e5-a2a033a0adbd None None] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:28:23.554915 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a587b1b9-29ca-4623-b6e5-a2a033a0adbd None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:28:23.555191 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a587b1b9-29ca-4623-b6e5-a2a033a0adbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:28:23.559490 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f94a97-b873-4cd4-b5bf-e4f052e38456 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:28:23.576880 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.auth.core [None req-a587b1b9-29ca-4623-b6e5-a2a033a0adbd None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80312) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:28:23.578333 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-a587b1b9-29ca-4623-b6e5-a2a033a0adbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:28:23.603737 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=100 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:28:23.619343 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7742b414-0495-4dd1-8916-6f4a6526728a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:28:23.619949 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7742b414-0495-4dd1-8916-6f4a6526728a None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:28:23.629044 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7742b414-0495-4dd1-8916-6f4a6526728a None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:28:23.629044 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7742b414-0495-4dd1-8916-6f4a6526728a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:28:23.636550 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7742b414-0495-4dd1-8916-6f4a6526728a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:28:23.638035 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7742b414-0495-4dd1-8916-6f4a6526728a None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:28:23.660794 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application [None req-7742b414-0495-4dd1-8916-6f4a6526728a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 16:28:23.660794 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 16:28:23.660794 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 16:28:23.660794 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 16:28:23.660794 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 16:28:23.660794 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 16:28:23.660794 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 16:28:23.660794 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 16:28:23.660794 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 16:28:23.660794 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 16:28:23.660794 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 16:28:23.660794 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 16:28:23.660794 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 01 16:28:23.660794 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 01 16:28:23.660794 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 01 16:28:23.662078 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 01 16:28:23.662078 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 16:28:23.662078 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 16:28:23.662078 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 16:28:23.662078 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 16:28:23.662078 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 16:28:23.662078 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 16:28:23.662078 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 16:28:23.662078 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application with Lock( Jul 01 16:28:23.662078 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 16:28:23.662078 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._enter() Jul 01 16:28:23.662078 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 16:28:23.662078 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 16:28:23.662078 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 16:28:23.662078 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.creator() Jul 01 16:28:23.662078 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 16:28:23.662078 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application created_value = creator( Jul 01 16:28:23.663334 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 01 16:28:23.663334 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 01 16:28:23.663334 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 16:28:23.663334 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Jul 01 16:28:23.663334 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 484/975] 10.222.0.57 () {56 vars in 1108 bytes} [Fri Jul 1 16:28:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 16:28:23.663997 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7742b414-0495-4dd1-8916-6f4a6526728a None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:28:23.677878 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 492/976] 10.222.0.57 () {58 vars in 1023 bytes} [Fri Jul 1 16:28:23 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 01 16:28:23.679234 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a587b1b9-29ca-4623-b6e5-a2a033a0adbd None None] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:28:23.783468 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7742b414-0495-4dd1-8916-6f4a6526728a None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:28:23.785918 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:28:23.803970 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a587b1b9-29ca-4623-b6e5-a2a033a0adbd None None] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:28:23.821311 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514e66c5-a88d-421e-bb0a-2933a0f1f4a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:28:23.821968 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514e66c5-a88d-421e-bb0a-2933a0f1f4a2 None admin] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:28:23.822253 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514e66c5-a88d-421e-bb0a-2933a0f1f4a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:28:23.822562 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514e66c5-a88d-421e-bb0a-2933a0f1f4a2 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:28:23.823299 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-514e66c5-a88d-421e-bb0a-2933a0f1f4a2 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:28:23.824661 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-514e66c5-a88d-421e-bb0a-2933a0f1f4a2 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:28:23.841021 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b0cf66-8ae6-40f8-a8ec-d84f3452e153 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:28:23.841695 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b0cf66-8ae6-40f8-a8ec-d84f3452e153 None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:28:23.841990 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b0cf66-8ae6-40f8-a8ec-d84f3452e153 None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:28:23.842263 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b0cf66-8ae6-40f8-a8ec-d84f3452e153 None admin] PATH_INFO: `/v3/domains` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:28:23.843027 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b0cf66-8ae6-40f8-a8ec-d84f3452e153 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80312) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:28:23.843344 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b0cf66-8ae6-40f8-a8ec-d84f3452e153 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:28:23.854596 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 485/977] 10.222.0.57 () {60 vars in 1144 bytes} [Fri Jul 1 16:28:23 2022] POST /identity/v3/endpoints => generated 349 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 16:28:23.854596 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514e66c5-a88d-421e-bb0a-2933a0f1f4a2 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:28:23.855420 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b0cf66-8ae6-40f8-a8ec-d84f3452e153 None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:28:23.859050 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 493/978] 10.222.0.57 () {56 vars in 1109 bytes} [Fri Jul 1 16:28:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:28:23.865525 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b0cf66-8ae6-40f8-a8ec-d84f3452e153 None admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:28:23.886775 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=100 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:28:23.995765 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514e66c5-a88d-421e-bb0a-2933a0f1f4a2 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:28:23.998236 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b0cf66-8ae6-40f8-a8ec-d84f3452e153 None admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:28:24.039541 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34aa590-41ae-4382-8355-999e6a7d76aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:28:24.040531 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34aa590-41ae-4382-8355-999e6a7d76aa None admin] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:28:24.041001 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34aa590-41ae-4382-8355-999e6a7d76aa None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:28:24.041470 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34aa590-41ae-4382-8355-999e6a7d76aa None admin] PATH_INFO: `/v3/services` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:28:24.042391 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194525d4-ae5f-45b5-8a6b-efc7847c4832 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:28:24.042610 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a34aa590-41ae-4382-8355-999e6a7d76aa None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:28:24.043241 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194525d4-ae5f-45b5-8a6b-efc7847c4832 None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:28:24.043521 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194525d4-ae5f-45b5-8a6b-efc7847c4832 None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:28:24.043797 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194525d4-ae5f-45b5-8a6b-efc7847c4832 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:28:24.044689 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a34aa590-41ae-4382-8355-999e6a7d76aa None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:28:24.055109 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-194525d4-ae5f-45b5-8a6b-efc7847c4832 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:28:24.057163 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-194525d4-ae5f-45b5-8a6b-efc7847c4832 None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:28:24.060357 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 486/979] 10.222.0.57 () {60 vars in 1141 bytes} [Fri Jul 1 16:28:23 2022] POST /identity/v3/services => generated 242 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 16:28:24.078760 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application [None req-194525d4-ae5f-45b5-8a6b-efc7847c4832 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 01 16:28:24.078760 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 16:28:24.078760 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 16:28:24.078760 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 16:28:24.078760 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 16:28:24.078760 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 16:28:24.078760 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 16:28:24.078760 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 16:28:24.078760 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 16:28:24.078760 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 16:28:24.078760 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 16:28:24.078760 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 16:28:24.078760 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 01 16:28:24.078760 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 01 16:28:24.078760 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 01 16:28:24.079919 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 01 16:28:24.079919 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 16:28:24.079919 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 16:28:24.079919 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 16:28:24.079919 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 16:28:24.079919 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 16:28:24.079919 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 16:28:24.079919 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 16:28:24.079919 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application with Lock( Jul 01 16:28:24.079919 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 16:28:24.079919 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self._enter() Jul 01 16:28:24.079919 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 16:28:24.079919 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 16:28:24.079919 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 16:28:24.079919 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.creator() Jul 01 16:28:24.079919 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 16:28:24.079919 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application created_value = creator( Jul 01 16:28:24.081102 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 01 16:28:24.081102 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 01 16:28:24.081102 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 01 16:28:24.081102 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 01 16:28:24.081102 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 01 16:28:24.081102 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 01 16:28:24.081102 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 01 16:28:24.081102 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application Jul 01 16:28:24.081102 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 494/980] 10.222.0.57 () {56 vars in 1111 bytes} [Fri Jul 1 16:28:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 01 16:28:24.087131 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34aa590-41ae-4382-8355-999e6a7d76aa None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:28:24.106734 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:28:24.236688 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34aa590-41ae-4382-8355-999e6a7d76aa None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:28:24.275263 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55542148-3860-4572-aff2-cbaf3627e08e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:28:24.275934 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55542148-3860-4572-aff2-cbaf3627e08e None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:28:24.276226 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55542148-3860-4572-aff2-cbaf3627e08e None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:28:24.276525 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55542148-3860-4572-aff2-cbaf3627e08e None admin] PATH_INFO: `/v3/projects` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:28:24.277277 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55542148-3860-4572-aff2-cbaf3627e08e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:28:24.277596 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55542148-3860-4572-aff2-cbaf3627e08e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:28:24.279073 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55542148-3860-4572-aff2-cbaf3627e08e None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:28:24.292011 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 487/981] 10.222.0.57 () {56 vars in 1148 bytes} [Fri Jul 1 16:28:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:28:24.298145 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194525d4-ae5f-45b5-8a6b-efc7847c4832 None admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:28:24.387382 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=100 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:28:24.414325 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194525d4-ae5f-45b5-8a6b-efc7847c4832 None admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:28:24.452738 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4752e83a-d44b-4e25-9ecc-81e4dc638f7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:28:24.453418 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4752e83a-d44b-4e25-9ecc-81e4dc638f7e None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:28:24.453704 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4752e83a-d44b-4e25-9ecc-81e4dc638f7e None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:28:24.453981 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4752e83a-d44b-4e25-9ecc-81e4dc638f7e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:28:24.454850 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4752e83a-d44b-4e25-9ecc-81e4dc638f7e None admin] RBAC: Adding query filter params (role.id=c3a62bbbb749474aad6655e693c8963d, scope.project.id=e0788e6725a84e57a4174284c1845755, user.id=c4de1b1eeaed4fc5ae7200b2c7a90957) {{(pid=80312) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:28:24.455168 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4752e83a-d44b-4e25-9ecc-81e4dc638f7e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:28:24.456616 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4752e83a-d44b-4e25-9ecc-81e4dc638f7e None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:28:24.469480 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 495/982] 10.222.0.57 () {56 vars in 1374 bytes} [Fri Jul 1 16:28:24 2022] GET /identity/v3/role_assignments?user.id=c4de1b1eeaed4fc5ae7200b2c7a90957&scope.project.id=e0788e6725a84e57a4174284c1845755&role.id=c3a62bbbb749474aad6655e693c8963d => generated 608 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:28:25.975533 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e656aa-9d72-4993-b1b4-4df805fb5ae5 None None] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:28:25.976048 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e656aa-9d72-4993-b1b4-4df805fb5ae5 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:28:25.976127 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e656aa-9d72-4993-b1b4-4df805fb5ae5 None None] PATH_INFO: `/` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:28:25.977468 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 488/983] 10.222.0.57 () {54 vars in 926 bytes} [Fri Jul 1 16:28:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 16:28:25.982741 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e532bd-5e6b-43c1-8bfb-8372ee8919d2 None None] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:28:25.983164 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e532bd-5e6b-43c1-8bfb-8372ee8919d2 None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:28:25.983329 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e532bd-5e6b-43c1-8bfb-8372ee8919d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:28:26.000876 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.auth.core [None req-e3e532bd-5e6b-43c1-8bfb-8372ee8919d2 None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80312) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:28:26.002065 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-e3e532bd-5e6b-43c1-8bfb-8372ee8919d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:28:26.074847 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 496/984] 10.222.0.57 () {58 vars in 1023 bytes} [Fri Jul 1 16:28:25 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 01 16:28:26.098726 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e656aa-9d72-4993-b1b4-4df805fb5ae5 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:28:26.171645 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:28:26.205666 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e656aa-9d72-4993-b1b4-4df805fb5ae5 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:28:26.212666 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc635e6b-9bf2-4d0f-93e7-e3af9718e31d None None] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:28:26.212958 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc635e6b-9bf2-4d0f-93e7-e3af9718e31d None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:28:26.213238 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc635e6b-9bf2-4d0f-93e7-e3af9718e31d None None] PATH_INFO: `/` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:28:26.214795 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 497/985] 10.222.0.57 () {54 vars in 926 bytes} [Fri Jul 1 16:28:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 16:28:26.219629 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa05d009-59f2-4ed4-839a-9c00ca11283f None None] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:28:26.219895 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa05d009-59f2-4ed4-839a-9c00ca11283f None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:28:26.220186 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa05d009-59f2-4ed4-839a-9c00ca11283f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:28:26.239695 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.auth.core [None req-fa05d009-59f2-4ed4-839a-9c00ca11283f None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80312) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:28:26.240887 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-fa05d009-59f2-4ed4-839a-9c00ca11283f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:28:26.244428 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef55f7f3-c044-47d0-b238-9e662ce257d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:28:26.245240 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef55f7f3-c044-47d0-b238-9e662ce257d9 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:28:26.245524 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef55f7f3-c044-47d0-b238-9e662ce257d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:28:26.245806 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef55f7f3-c044-47d0-b238-9e662ce257d9 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:28:26.246519 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef55f7f3-c044-47d0-b238-9e662ce257d9 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:28:26.247871 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef55f7f3-c044-47d0-b238-9e662ce257d9 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:28:26.260684 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application [None req-ef55f7f3-c044-47d0-b238-9e662ce257d9 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 01 16:28:26.260684 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 16:28:26.260684 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 01 16:28:26.260684 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 01 16:28:26.260684 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 01 16:28:26.260684 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 01 16:28:26.260684 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 01 16:28:26.260684 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 01 16:28:26.260684 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 01 16:28:26.260684 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Jul 01 16:28:26.260684 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 01 16:28:26.260684 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Jul 01 16:28:26.260684 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 16:28:26.260684 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 16:28:26.260684 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 16:28:26.260684 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 16:28:26.260684 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 16:28:26.261665 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 16:28:26.261665 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 16:28:26.261665 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 16:28:26.261665 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 16:28:26.261665 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 16:28:26.261665 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 16:28:26.261665 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 01 16:28:26.261665 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 01 16:28:26.261665 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 01 16:28:26.261665 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 01 16:28:26.261665 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 16:28:26.261665 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 16:28:26.261665 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 16:28:26.261665 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 16:28:26.261665 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 16:28:26.261665 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 16:28:26.262642 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 16:28:26.262642 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application with Lock( Jul 01 16:28:26.262642 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 16:28:26.262642 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._enter() Jul 01 16:28:26.262642 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 16:28:26.262642 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 16:28:26.262642 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 16:28:26.262642 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.creator() Jul 01 16:28:26.262642 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 16:28:26.262642 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application created_value = creator( Jul 01 16:28:26.262642 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 01 16:28:26.262642 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 01 16:28:26.262642 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 01 16:28:26.262642 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Jul 01 16:28:26.262642 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 489/986] 10.222.0.57 () {56 vars in 1126 bytes} [Fri Jul 1 16:28:26 2022] GET /identity/v3/services/object-store => generated 93 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 16:28:26.267781 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef55f7f3-c044-47d0-b238-9e662ce257d9 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:28:26.309917 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=100 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:28:26.316249 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 498/987] 10.222.0.57 () {58 vars in 1023 bytes} [Fri Jul 1 16:28:26 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 01 16:28:26.340795 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa05d009-59f2-4ed4-839a-9c00ca11283f None None] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:28:26.385505 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef55f7f3-c044-47d0-b238-9e662ce257d9 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:28:26.427056 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd6f23f-0d6f-4945-a563-8be0178ce9b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:28:26.427726 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd6f23f-0d6f-4945-a563-8be0178ce9b2 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:28:26.428013 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd6f23f-0d6f-4945-a563-8be0178ce9b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:28:26.428303 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd6f23f-0d6f-4945-a563-8be0178ce9b2 None admin] PATH_INFO: `/v3/services` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:28:26.429008 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd6f23f-0d6f-4945-a563-8be0178ce9b2 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:28:26.429318 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd6f23f-0d6f-4945-a563-8be0178ce9b2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:28:26.430717 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd6f23f-0d6f-4945-a563-8be0178ce9b2 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:28:26.431827 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:28:26.438152 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 490/988] 10.222.0.57 () {56 vars in 1122 bytes} [Fri Jul 1 16:28:26 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 01 16:28:26.443681 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd6f23f-0d6f-4945-a563-8be0178ce9b2 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:28:26.451437 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa05d009-59f2-4ed4-839a-9c00ca11283f None None] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:28:26.490819 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f380f4-3cf5-4452-ab30-5fa41a4be6de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:28:26.491977 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f380f4-3cf5-4452-ab30-5fa41a4be6de None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:28:26.492269 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f380f4-3cf5-4452-ab30-5fa41a4be6de None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:28:26.492540 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f380f4-3cf5-4452-ab30-5fa41a4be6de None admin] PATH_INFO: `/v3/services/compute` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:28:26.493615 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f380f4-3cf5-4452-ab30-5fa41a4be6de None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:28:26.494936 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f380f4-3cf5-4452-ab30-5fa41a4be6de None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:28:26.507593 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application [None req-50f380f4-3cf5-4452-ab30-5fa41a4be6de None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 01 16:28:26.507593 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 16:28:26.507593 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 01 16:28:26.507593 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 01 16:28:26.507593 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 01 16:28:26.507593 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 01 16:28:26.507593 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 01 16:28:26.507593 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 01 16:28:26.507593 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 01 16:28:26.507593 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application Jul 01 16:28:26.507593 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 01 16:28:26.507593 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application Jul 01 16:28:26.507593 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 16:28:26.507593 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 16:28:26.507593 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 16:28:26.507593 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 16:28:26.507593 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 16:28:26.508641 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 16:28:26.508641 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 16:28:26.508641 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 16:28:26.508641 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 16:28:26.508641 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 16:28:26.508641 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 16:28:26.508641 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 01 16:28:26.508641 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 01 16:28:26.508641 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 01 16:28:26.508641 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 01 16:28:26.508641 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 16:28:26.508641 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 16:28:26.508641 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 16:28:26.508641 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 16:28:26.508641 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 16:28:26.508641 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 16:28:26.509522 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 16:28:26.509522 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application with Lock( Jul 01 16:28:26.509522 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 16:28:26.509522 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self._enter() Jul 01 16:28:26.509522 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 16:28:26.509522 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 16:28:26.509522 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 16:28:26.509522 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.creator() Jul 01 16:28:26.509522 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 16:28:26.509522 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application created_value = creator( Jul 01 16:28:26.509522 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 01 16:28:26.509522 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 01 16:28:26.509522 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 01 16:28:26.509522 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application Jul 01 16:28:26.509522 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 499/989] 10.222.0.57 () {56 vars in 1111 bytes} [Fri Jul 1 16:28:26 2022] GET /identity/v3/services/compute => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 01 16:28:26.515894 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f380f4-3cf5-4452-ab30-5fa41a4be6de None admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:28:26.560915 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd6f23f-0d6f-4945-a563-8be0178ce9b2 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:28:26.578160 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=100 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:28:26.595818 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc389346-b598-4afb-9191-4e99d534da25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:28:26.596473 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc389346-b598-4afb-9191-4e99d534da25 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:28:26.596756 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc389346-b598-4afb-9191-4e99d534da25 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:28:26.597032 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc389346-b598-4afb-9191-4e99d534da25 None admin] PATH_INFO: `/v3/services` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:28:26.597954 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc389346-b598-4afb-9191-4e99d534da25 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:28:26.598508 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc389346-b598-4afb-9191-4e99d534da25 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:28:26.599874 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc389346-b598-4afb-9191-4e99d534da25 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:28:26.609354 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 491/990] 10.222.0.57 () {56 vars in 1122 bytes} [Fri Jul 1 16:28:26 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 01 16:28:26.609354 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c389e7f-80a4-4b3e-845c-cd9fd755db7c None None] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:28:26.609354 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c389e7f-80a4-4b3e-845c-cd9fd755db7c None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:28:26.609354 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c389e7f-80a4-4b3e-845c-cd9fd755db7c None None] PATH_INFO: `/` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:28:26.609354 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 492/991] 10.222.0.57 () {54 vars in 926 bytes} [Fri Jul 1 16:28:26 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 16:28:26.611123 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c389e7f-80a4-4b3e-845c-cd9fd755db7c None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:28:26.619386 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f380f4-3cf5-4452-ab30-5fa41a4be6de None admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:28:26.654256 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b66dca-93b0-4d9c-b9fb-d51e31a2b085 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:28:26.654873 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b66dca-93b0-4d9c-b9fb-d51e31a2b085 None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:28:26.655157 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b66dca-93b0-4d9c-b9fb-d51e31a2b085 None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:28:26.655432 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b66dca-93b0-4d9c-b9fb-d51e31a2b085 None admin] PATH_INFO: `/v3/services` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:28:26.656134 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b66dca-93b0-4d9c-b9fb-d51e31a2b085 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80312) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:28:26.656440 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b66dca-93b0-4d9c-b9fb-d51e31a2b085 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:28:26.657797 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b66dca-93b0-4d9c-b9fb-d51e31a2b085 None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:28:26.667491 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 500/992] 10.222.0.57 () {56 vars in 1112 bytes} [Fri Jul 1 16:28:26 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:28:26.667491 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd679c3-ebde-419b-a740-a878584b6b6e None None] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:28:26.667491 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd679c3-ebde-419b-a740-a878584b6b6e None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:28:26.667491 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd679c3-ebde-419b-a740-a878584b6b6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:28:26.675097 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:28:26.682870 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.auth.core [None req-edd679c3-ebde-419b-a740-a878584b6b6e None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80312) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:28:26.684074 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-edd679c3-ebde-419b-a740-a878584b6b6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:28:26.722253 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c389e7f-80a4-4b3e-845c-cd9fd755db7c None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:28:26.757304 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c26b2f-ff90-40e9-b613-b2183e3e6e9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:28:26.757962 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c26b2f-ff90-40e9-b613-b2183e3e6e9e None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:28:26.758250 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c26b2f-ff90-40e9-b613-b2183e3e6e9e None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:28:26.758565 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c26b2f-ff90-40e9-b613-b2183e3e6e9e None admin] PATH_INFO: `/v3/endpoints` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:28:26.759361 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c26b2f-ff90-40e9-b613-b2183e3e6e9e None admin] RBAC: Adding query filter params (interface=admin, service_id=869b98028e3547e3b6563b17f76e37cb, region_id=RegionOne) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:28:26.759678 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c26b2f-ff90-40e9-b613-b2183e3e6e9e None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:28:26.761051 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c26b2f-ff90-40e9-b613-b2183e3e6e9e None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:28:26.762234 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 501/993] 10.222.0.57 () {58 vars in 1023 bytes} [Fri Jul 1 16:28:26 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 01 16:28:26.774118 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd679c3-ebde-419b-a740-a878584b6b6e None None] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:28:26.774244 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 493/994] 10.222.0.57 () {56 vars in 1249 bytes} [Fri Jul 1 16:28:26 2022] GET /identity/v3/endpoints?service_id=869b98028e3547e3b6563b17f76e37cb&interface=admin®ion_id=RegionOne => generated 193 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 01 16:28:26.774813 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c26b2f-ff90-40e9-b613-b2183e3e6e9e None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:28:26.819943 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=100 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:28:26.870533 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd679c3-ebde-419b-a740-a878584b6b6e None None] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:28:26.876282 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c26b2f-ff90-40e9-b613-b2183e3e6e9e None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:28:26.902230 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:28:26.905491 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3180c8-514b-4df5-9731-b5c76f196461 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:28:26.906133 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3180c8-514b-4df5-9731-b5c76f196461 None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:28:26.906453 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3180c8-514b-4df5-9731-b5c76f196461 None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:28:26.906743 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3180c8-514b-4df5-9731-b5c76f196461 None admin] PATH_INFO: `/v3/services` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:28:26.907458 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f3180c8-514b-4df5-9731-b5c76f196461 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80312) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:28:26.907769 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f3180c8-514b-4df5-9731-b5c76f196461 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:28:26.909152 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f3180c8-514b-4df5-9731-b5c76f196461 None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:28:26.912211 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376cf0fd-3756-4beb-9696-03196bb0885c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:28:26.912878 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376cf0fd-3756-4beb-9696-03196bb0885c None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:28:26.913161 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376cf0fd-3756-4beb-9696-03196bb0885c None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:28:26.913457 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376cf0fd-3756-4beb-9696-03196bb0885c None admin] PATH_INFO: `/v3/services` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:28:26.914137 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-376cf0fd-3756-4beb-9696-03196bb0885c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:28:26.915523 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-376cf0fd-3756-4beb-9696-03196bb0885c None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:28:26.916606 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 502/995] 10.222.0.57 () {56 vars in 1112 bytes} [Fri Jul 1 16:28:26 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:28:26.917874 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3180c8-514b-4df5-9731-b5c76f196461 None admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:28:26.924430 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 494/996] 10.222.0.57 () {56 vars in 1087 bytes} [Fri Jul 1 16:28:26 2022] GET /identity/v3/services => generated 1727 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 01 16:28:26.925967 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376cf0fd-3756-4beb-9696-03196bb0885c None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:28:27.034604 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3180c8-514b-4df5-9731-b5c76f196461 None admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:28:27.051132 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376cf0fd-3756-4beb-9696-03196bb0885c None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:28:27.056683 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=100 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:28:27.074422 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2abaa9-c37d-4281-ba17-32f913f2373b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:28:27.075230 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2abaa9-c37d-4281-ba17-32f913f2373b None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:28:27.075522 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2abaa9-c37d-4281-ba17-32f913f2373b None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:28:27.075794 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2abaa9-c37d-4281-ba17-32f913f2373b None admin] PATH_INFO: `/v3/services/image` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:28:27.076460 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f2abaa9-c37d-4281-ba17-32f913f2373b None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:28:27.078138 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f2abaa9-c37d-4281-ba17-32f913f2373b None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:28:27.091034 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application [None req-9f2abaa9-c37d-4281-ba17-32f913f2373b None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 01 16:28:27.091034 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 16:28:27.091034 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 01 16:28:27.091034 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 01 16:28:27.091034 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 01 16:28:27.091034 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 01 16:28:27.091034 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 01 16:28:27.091034 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 01 16:28:27.091034 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 01 16:28:27.091034 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application Jul 01 16:28:27.091034 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 01 16:28:27.091034 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application Jul 01 16:28:27.091034 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 16:28:27.091034 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 16:28:27.091034 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 16:28:27.091034 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 16:28:27.091034 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 16:28:27.092061 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 16:28:27.092061 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 16:28:27.092061 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 16:28:27.092061 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 16:28:27.092061 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 16:28:27.092061 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 16:28:27.092061 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 01 16:28:27.092061 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 01 16:28:27.092061 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 01 16:28:27.092061 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 01 16:28:27.092061 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 16:28:27.092061 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 16:28:27.092061 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 16:28:27.092061 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 16:28:27.092061 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 16:28:27.092061 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 16:28:27.092910 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 16:28:27.092910 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application with Lock( Jul 01 16:28:27.092910 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 16:28:27.092910 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self._enter() Jul 01 16:28:27.092910 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 16:28:27.092910 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 16:28:27.092910 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 16:28:27.092910 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.creator() Jul 01 16:28:27.092910 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 16:28:27.092910 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application created_value = creator( Jul 01 16:28:27.092910 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 01 16:28:27.092910 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 01 16:28:27.092910 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 01 16:28:27.092910 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application Jul 01 16:28:27.092910 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 503/997] 10.222.0.57 () {56 vars in 1105 bytes} [Fri Jul 1 16:28:26 2022] GET /identity/v3/services/image => generated 86 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 16:28:27.096025 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5b6f94-e4a2-4d52-b029-7d36895f7136 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:28:27.096682 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5b6f94-e4a2-4d52-b029-7d36895f7136 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:28:27.097063 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5b6f94-e4a2-4d52-b029-7d36895f7136 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:28:27.097428 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5b6f94-e4a2-4d52-b029-7d36895f7136 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:28:27.098613 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c5b6f94-e4a2-4d52-b029-7d36895f7136 None admin] RBAC: Adding query filter params (interface=public, service_id=6128b9eb1fa24d838d963e35999971fc, region_id=RegionOne) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:28:27.098936 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c5b6f94-e4a2-4d52-b029-7d36895f7136 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:28:27.099718 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2abaa9-c37d-4281-ba17-32f913f2373b None admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:28:27.100317 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c5b6f94-e4a2-4d52-b029-7d36895f7136 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:28:27.106974 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 495/998] 10.222.0.57 () {56 vars in 1251 bytes} [Fri Jul 1 16:28:26 2022] GET /identity/v3/endpoints?service_id=6128b9eb1fa24d838d963e35999971fc&interface=public®ion_id=RegionOne => generated 194 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:28:27.113502 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5b6f94-e4a2-4d52-b029-7d36895f7136 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:28:27.172000 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:28:27.212602 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2abaa9-c37d-4281-ba17-32f913f2373b None admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:28:27.220642 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5b6f94-e4a2-4d52-b029-7d36895f7136 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:28:27.251581 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23acd769-b831-4a82-b790-aa47f2b47532 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:28:27.252398 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23acd769-b831-4a82-b790-aa47f2b47532 None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:28:27.252683 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23acd769-b831-4a82-b790-aa47f2b47532 None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:28:27.252958 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23acd769-b831-4a82-b790-aa47f2b47532 None admin] PATH_INFO: `/v3/services` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:28:27.254297 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23acd769-b831-4a82-b790-aa47f2b47532 None admin] RBAC: Adding query filter params (name=image) {{(pid=80312) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:28:27.254773 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23acd769-b831-4a82-b790-aa47f2b47532 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:28:27.256125 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23acd769-b831-4a82-b790-aa47f2b47532 None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:28:27.256311 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8109905-9d6f-42a7-8bce-2ccdad45774f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:28:27.256959 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8109905-9d6f-42a7-8bce-2ccdad45774f None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:28:27.257253 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8109905-9d6f-42a7-8bce-2ccdad45774f None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:28:27.257539 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8109905-9d6f-42a7-8bce-2ccdad45774f None admin] PATH_INFO: `/v3/services` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:28:27.258222 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8109905-9d6f-42a7-8bce-2ccdad45774f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:28:27.259625 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8109905-9d6f-42a7-8bce-2ccdad45774f None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:28:27.262155 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 504/999] 10.222.0.57 () {56 vars in 1108 bytes} [Fri Jul 1 16:28:27 2022] GET /identity/v3/services?name=image => generated 122 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:28:27.266430 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 496/1000] 10.222.0.57 () {56 vars in 1087 bytes} [Fri Jul 1 16:28:27 2022] GET /identity/v3/services => generated 1727 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 16:28:27.271747 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23acd769-b831-4a82-b790-aa47f2b47532 None admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:28:27.328117 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=100 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:28:27.385280 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23acd769-b831-4a82-b790-aa47f2b47532 None admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:28:27.422315 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d66c09-5ed3-4757-8338-7273c0254df5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:28:27.422940 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d66c09-5ed3-4757-8338-7273c0254df5 None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:28:27.423231 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d66c09-5ed3-4757-8338-7273c0254df5 None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:28:27.423508 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d66c09-5ed3-4757-8338-7273c0254df5 None admin] PATH_INFO: `/v3/services` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:28:27.424222 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d66c09-5ed3-4757-8338-7273c0254df5 None admin] RBAC: Adding query filter params (type=image) {{(pid=80312) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:28:27.424535 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d66c09-5ed3-4757-8338-7273c0254df5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:28:27.426778 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d66c09-5ed3-4757-8338-7273c0254df5 None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:28:27.431747 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 505/1001] 10.222.0.57 () {56 vars in 1108 bytes} [Fri Jul 1 16:28:27 2022] GET /identity/v3/services?type=image => generated 122 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:28:28.977296 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20888b26-766f-436c-b039-5a253169e1d6 None None] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:28:28.977810 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20888b26-766f-436c-b039-5a253169e1d6 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:28:28.977926 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20888b26-766f-436c-b039-5a253169e1d6 None None] PATH_INFO: `/` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:28:28.979300 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 497/1002] 10.222.0.57 () {54 vars in 926 bytes} [Fri Jul 1 16:28:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 16:28:28.984475 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f497de58-79a1-4deb-846d-9feaacfcad09 None None] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:28:28.984894 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f497de58-79a1-4deb-846d-9feaacfcad09 None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:28:28.985056 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f497de58-79a1-4deb-846d-9feaacfcad09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:28:29.001483 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.auth.core [None req-f497de58-79a1-4deb-846d-9feaacfcad09 None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80312) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:28:29.002689 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-f497de58-79a1-4deb-846d-9feaacfcad09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:28:29.072259 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 506/1003] 10.222.0.57 () {58 vars in 1023 bytes} [Fri Jul 1 16:28:28 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 01 16:28:29.095595 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20888b26-766f-436c-b039-5a253169e1d6 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:28:29.196482 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20888b26-766f-436c-b039-5a253169e1d6 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:28:29.214379 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:28:29.231332 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f20db52-c82e-4af1-87a3-d93acaab1f83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:28:29.231983 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f20db52-c82e-4af1-87a3-d93acaab1f83 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:28:29.232263 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f20db52-c82e-4af1-87a3-d93acaab1f83 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:28:29.232538 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f20db52-c82e-4af1-87a3-d93acaab1f83 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:28:29.233187 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f20db52-c82e-4af1-87a3-d93acaab1f83 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:28:29.234575 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f20db52-c82e-4af1-87a3-d93acaab1f83 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:28:29.247391 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application [None req-2f20db52-c82e-4af1-87a3-d93acaab1f83 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 01 16:28:29.247391 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 16:28:29.247391 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 01 16:28:29.247391 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 01 16:28:29.247391 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 01 16:28:29.247391 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 01 16:28:29.247391 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 01 16:28:29.247391 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 01 16:28:29.247391 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 01 16:28:29.247391 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Jul 01 16:28:29.247391 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 01 16:28:29.247391 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Jul 01 16:28:29.247391 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 16:28:29.247391 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 16:28:29.247391 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 16:28:29.247391 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 16:28:29.247391 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 16:28:29.248492 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 16:28:29.248492 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 16:28:29.248492 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 16:28:29.248492 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 16:28:29.248492 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 16:28:29.248492 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 16:28:29.248492 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 01 16:28:29.248492 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 01 16:28:29.248492 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 01 16:28:29.248492 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 01 16:28:29.248492 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 16:28:29.248492 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 16:28:29.248492 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 16:28:29.248492 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 16:28:29.248492 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 16:28:29.248492 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 16:28:29.249339 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 16:28:29.249339 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application with Lock( Jul 01 16:28:29.249339 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 16:28:29.249339 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._enter() Jul 01 16:28:29.249339 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 16:28:29.249339 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 16:28:29.249339 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 16:28:29.249339 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.creator() Jul 01 16:28:29.249339 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 16:28:29.249339 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application created_value = creator( Jul 01 16:28:29.249339 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 01 16:28:29.249339 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 01 16:28:29.249339 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 01 16:28:29.249339 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Jul 01 16:28:29.249339 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 498/1004] 10.222.0.57 () {56 vars in 1126 bytes} [Fri Jul 1 16:28:29 2022] GET /identity/v3/services/object-store => generated 93 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 16:28:29.256807 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f497de58-79a1-4deb-846d-9feaacfcad09 None None] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:28:29.298507 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=100 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:28:29.357723 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1569973e-3e3a-435a-9fc8-5fdc1cb6dabc None None] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:28:29.357917 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f497de58-79a1-4deb-846d-9feaacfcad09 None None] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:28:29.358044 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1569973e-3e3a-435a-9fc8-5fdc1cb6dabc None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:28:29.358285 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1569973e-3e3a-435a-9fc8-5fdc1cb6dabc None None] PATH_INFO: `/` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:28:29.359842 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 499/1005] 10.222.0.57 () {54 vars in 926 bytes} [Fri Jul 1 16:28:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 16:28:29.366679 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9dc251c-1b95-4e43-ba07-40f1e3162702 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:28:29.367199 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9dc251c-1b95-4e43-ba07-40f1e3162702 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:28:29.367676 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9dc251c-1b95-4e43-ba07-40f1e3162702 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:28:29.386926 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-f9dc251c-1b95-4e43-ba07-40f1e3162702 None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:28:29.388510 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-f9dc251c-1b95-4e43-ba07-40f1e3162702 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:28:29.399962 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4768394-4a29-4ebf-9556-a9a626299e6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:28:29.400644 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4768394-4a29-4ebf-9556-a9a626299e6e None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:28:29.400932 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4768394-4a29-4ebf-9556-a9a626299e6e None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:28:29.401208 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4768394-4a29-4ebf-9556-a9a626299e6e None admin] PATH_INFO: `/v3/services` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:28:29.401925 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4768394-4a29-4ebf-9556-a9a626299e6e None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80312) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:28:29.402239 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4768394-4a29-4ebf-9556-a9a626299e6e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:28:29.404170 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4768394-4a29-4ebf-9556-a9a626299e6e None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:28:29.408914 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 507/1006] 10.222.0.57 () {56 vars in 1122 bytes} [Fri Jul 1 16:28:29 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:28:29.414474 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4768394-4a29-4ebf-9556-a9a626299e6e None admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:28:29.464966 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 500/1007] 10.222.0.57 () {58 vars in 1023 bytes} [Fri Jul 1 16:28:29 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 01 16:28:29.488785 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9dc251c-1b95-4e43-ba07-40f1e3162702 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:28:29.523124 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4768394-4a29-4ebf-9556-a9a626299e6e None admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:28:29.530947 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=100 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:28:29.558108 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b922e1c-e973-4baa-b051-1a6e2529cb31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:28:29.558806 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b922e1c-e973-4baa-b051-1a6e2529cb31 None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:28:29.559096 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b922e1c-e973-4baa-b051-1a6e2529cb31 None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:28:29.559378 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b922e1c-e973-4baa-b051-1a6e2529cb31 None admin] PATH_INFO: `/v3/services` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:28:29.560105 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b922e1c-e973-4baa-b051-1a6e2529cb31 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80312) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:28:29.560400 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b922e1c-e973-4baa-b051-1a6e2529cb31 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:28:29.561769 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b922e1c-e973-4baa-b051-1a6e2529cb31 None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:28:29.567201 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 508/1008] 10.222.0.57 () {56 vars in 1122 bytes} [Fri Jul 1 16:28:29 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 01 16:28:29.568520 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204b79bb-dbf9-45bf-a32b-7fb6b5dc14a0 None None] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:28:29.568814 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204b79bb-dbf9-45bf-a32b-7fb6b5dc14a0 None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:28:29.569092 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204b79bb-dbf9-45bf-a32b-7fb6b5dc14a0 None None] PATH_INFO: `/` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:28:29.570360 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 509/1009] 10.222.0.57 () {54 vars in 926 bytes} [Fri Jul 1 16:28:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 16:28:29.572747 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204b79bb-dbf9-45bf-a32b-7fb6b5dc14a0 None None] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:28:29.591851 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9dc251c-1b95-4e43-ba07-40f1e3162702 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:28:29.604670 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:28:29.628270 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4b8bb7-4a09-459c-b0b9-8582f71cc58a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:28:29.629648 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4b8bb7-4a09-459c-b0b9-8582f71cc58a None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:28:29.629932 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4b8bb7-4a09-459c-b0b9-8582f71cc58a None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:28:29.630212 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4b8bb7-4a09-459c-b0b9-8582f71cc58a None admin] PATH_INFO: `/v3/services/compute` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:28:29.630919 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af4b8bb7-4a09-459c-b0b9-8582f71cc58a None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:28:29.632293 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af4b8bb7-4a09-459c-b0b9-8582f71cc58a None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:28:29.644629 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application [None req-af4b8bb7-4a09-459c-b0b9-8582f71cc58a None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 01 16:28:29.644629 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 16:28:29.644629 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 01 16:28:29.644629 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 01 16:28:29.644629 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 01 16:28:29.644629 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 01 16:28:29.644629 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 01 16:28:29.644629 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 01 16:28:29.644629 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 01 16:28:29.644629 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Jul 01 16:28:29.644629 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 01 16:28:29.644629 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Jul 01 16:28:29.644629 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 16:28:29.644629 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 16:28:29.644629 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 16:28:29.644629 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 16:28:29.644629 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 16:28:29.645774 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 16:28:29.645774 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 16:28:29.645774 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 16:28:29.645774 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 16:28:29.645774 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 16:28:29.645774 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 16:28:29.645774 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 01 16:28:29.645774 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 01 16:28:29.645774 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 01 16:28:29.645774 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 01 16:28:29.645774 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 16:28:29.645774 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 16:28:29.645774 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 16:28:29.645774 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 16:28:29.645774 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 16:28:29.645774 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 16:28:29.651850 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 16:28:29.651850 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application with Lock( Jul 01 16:28:29.651850 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 16:28:29.651850 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._enter() Jul 01 16:28:29.651850 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 16:28:29.651850 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 16:28:29.651850 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 16:28:29.651850 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.creator() Jul 01 16:28:29.651850 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 16:28:29.651850 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application created_value = creator( Jul 01 16:28:29.651850 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 01 16:28:29.651850 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 01 16:28:29.651850 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 01 16:28:29.651850 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Jul 01 16:28:29.651850 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 501/1010] 10.222.0.57 () {56 vars in 1111 bytes} [Fri Jul 1 16:28:29 2022] GET /identity/v3/services/compute => generated 88 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 01 16:28:29.652669 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b88388a-2bb8-4d46-b2cb-5f700e70c8a1 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:28:29.652669 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b88388a-2bb8-4d46-b2cb-5f700e70c8a1 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:28:29.652669 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b88388a-2bb8-4d46-b2cb-5f700e70c8a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:28:29.664198 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-2b88388a-2bb8-4d46-b2cb-5f700e70c8a1 None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:28:29.665370 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-2b88388a-2bb8-4d46-b2cb-5f700e70c8a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:28:29.676507 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204b79bb-dbf9-45bf-a32b-7fb6b5dc14a0 None None] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:28:29.713651 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102f26bd-3dc4-416c-87fc-83a3a6585a10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:28:29.714311 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102f26bd-3dc4-416c-87fc-83a3a6585a10 None admin] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:28:29.714626 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102f26bd-3dc4-416c-87fc-83a3a6585a10 None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:28:29.714903 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102f26bd-3dc4-416c-87fc-83a3a6585a10 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:28:29.715684 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-102f26bd-3dc4-416c-87fc-83a3a6585a10 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:28:29.716988 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-102f26bd-3dc4-416c-87fc-83a3a6585a10 None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:28:29.732162 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 510/1011] 10.222.0.57 () {60 vars in 1144 bytes} [Fri Jul 1 16:28:29 2022] POST /identity/v3/endpoints => generated 325 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 16:28:29.732688 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 502/1012] 10.222.0.57 () {58 vars in 1023 bytes} [Fri Jul 1 16:28:29 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 01 16:28:29.734954 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102f26bd-3dc4-416c-87fc-83a3a6585a10 None admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:28:29.757732 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b88388a-2bb8-4d46-b2cb-5f700e70c8a1 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:28:29.776170 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:28:29.877413 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b88388a-2bb8-4d46-b2cb-5f700e70c8a1 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:28:29.885277 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102f26bd-3dc4-416c-87fc-83a3a6585a10 None admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:28:29.885844 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:28:29.917737 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9767564-9245-49c5-851a-6026c1a4e04d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:28:29.917737 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9767564-9245-49c5-851a-6026c1a4e04d None admin] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:28:29.918124 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9767564-9245-49c5-851a-6026c1a4e04d None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:28:29.918255 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9767564-9245-49c5-851a-6026c1a4e04d None admin] PATH_INFO: `/v3/services` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:28:29.918964 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9767564-9245-49c5-851a-6026c1a4e04d None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:28:29.920327 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9767564-9245-49c5-851a-6026c1a4e04d None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:28:29.929504 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3ff48c-718f-4172-8970-32eb416e956e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:28:29.930697 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3ff48c-718f-4172-8970-32eb416e956e None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:28:29.931190 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3ff48c-718f-4172-8970-32eb416e956e None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:28:29.931682 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3ff48c-718f-4172-8970-32eb416e956e None admin] PATH_INFO: `/v3/services` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:28:29.932999 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3ff48c-718f-4172-8970-32eb416e956e None admin] RBAC: Adding query filter params (name=compute) {{(pid=80312) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:28:29.933586 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3ff48c-718f-4172-8970-32eb416e956e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:28:29.936147 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3ff48c-718f-4172-8970-32eb416e956e None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:28:29.938224 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 503/1013] 10.222.0.57 () {60 vars in 1141 bytes} [Fri Jul 1 16:28:29 2022] POST /identity/v3/services => generated 242 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 16:28:29.946368 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 511/1014] 10.222.0.57 () {56 vars in 1112 bytes} [Fri Jul 1 16:28:29 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:28:29.952815 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9767564-9245-49c5-851a-6026c1a4e04d None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:28:30.083934 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9767564-9245-49c5-851a-6026c1a4e04d None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:28:30.123131 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1912bb43-e937-4589-840a-c31ddf2e5647 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:28:30.123658 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1912bb43-e937-4589-840a-c31ddf2e5647 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:28:30.123945 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1912bb43-e937-4589-840a-c31ddf2e5647 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:28:30.124224 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1912bb43-e937-4589-840a-c31ddf2e5647 None admin] PATH_INFO: `/v3/services` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:28:30.124943 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1912bb43-e937-4589-840a-c31ddf2e5647 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:28:30.125255 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1912bb43-e937-4589-840a-c31ddf2e5647 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:28:30.126662 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1912bb43-e937-4589-840a-c31ddf2e5647 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:28:30.133201 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 504/1015] 10.222.0.57 () {56 vars in 1112 bytes} [Fri Jul 1 16:28:29 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:28:30.139913 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3ff48c-718f-4172-8970-32eb416e956e None admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:28:30.243026 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=100 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:28:30.256868 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3ff48c-718f-4172-8970-32eb416e956e None admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:28:30.294540 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63472bd3-691a-47f3-b8e7-ad02395b4181 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:28:30.295205 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63472bd3-691a-47f3-b8e7-ad02395b4181 None admin] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:28:30.295491 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63472bd3-691a-47f3-b8e7-ad02395b4181 None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:28:30.295770 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63472bd3-691a-47f3-b8e7-ad02395b4181 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:28:30.296532 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63472bd3-691a-47f3-b8e7-ad02395b4181 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:28:30.297905 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63472bd3-691a-47f3-b8e7-ad02395b4181 None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:28:30.326384 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 512/1016] 10.222.0.57 () {60 vars in 1144 bytes} [Fri Jul 1 16:28:30 2022] POST /identity/v3/endpoints => generated 334 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 16:28:31.870907 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec3fd41-1387-4ff8-b62c-1cdf8455dcb6 None None] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:28:31.871414 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec3fd41-1387-4ff8-b62c-1cdf8455dcb6 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:28:31.871525 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec3fd41-1387-4ff8-b62c-1cdf8455dcb6 None None] PATH_INFO: `/` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:28:31.873083 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 505/1017] 10.222.0.57 () {54 vars in 926 bytes} [Fri Jul 1 16:28:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 16:28:31.878427 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b8baca-287e-4b5f-8bc9-9030898013fb None None] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:28:31.879511 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b8baca-287e-4b5f-8bc9-9030898013fb None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:28:31.879617 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b8baca-287e-4b5f-8bc9-9030898013fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:28:31.898731 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.auth.core [None req-25b8baca-287e-4b5f-8bc9-9030898013fb None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80312) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:28:31.899929 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-25b8baca-287e-4b5f-8bc9-9030898013fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:28:31.980011 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 513/1018] 10.222.0.57 () {58 vars in 1023 bytes} [Fri Jul 1 16:28:31 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 01 16:28:32.005281 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec3fd41-1387-4ff8-b62c-1cdf8455dcb6 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:28:32.034211 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:28:32.062329 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a5c4be-4bc4-4050-bec5-5a6236ebf6a0 None None] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:28:32.062700 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a5c4be-4bc4-4050-bec5-5a6236ebf6a0 None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:28:32.062928 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a5c4be-4bc4-4050-bec5-5a6236ebf6a0 None None] PATH_INFO: `/` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:28:32.064453 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 514/1019] 10.222.0.57 () {54 vars in 926 bytes} [Fri Jul 1 16:28:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 01 16:28:32.069805 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1820b0ca-9a7c-4697-8afc-fe4d35810633 None None] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:28:32.070102 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1820b0ca-9a7c-4697-8afc-fe4d35810633 None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:28:32.070416 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1820b0ca-9a7c-4697-8afc-fe4d35810633 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:28:32.088616 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.auth.core [None req-1820b0ca-9a7c-4697-8afc-fe4d35810633 None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80312) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:28:32.089960 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-1820b0ca-9a7c-4697-8afc-fe4d35810633 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:28:32.116854 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec3fd41-1387-4ff8-b62c-1cdf8455dcb6 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:28:32.152775 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f257f4b-56ef-4386-b9a9-ec99c830c200 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:28:32.153560 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f257f4b-56ef-4386-b9a9-ec99c830c200 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:28:32.153845 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f257f4b-56ef-4386-b9a9-ec99c830c200 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:28:32.154126 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f257f4b-56ef-4386-b9a9-ec99c830c200 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:28:32.155334 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f257f4b-56ef-4386-b9a9-ec99c830c200 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:28:32.156759 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f257f4b-56ef-4386-b9a9-ec99c830c200 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:28:32.158434 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 506/1020] 10.222.0.57 () {56 vars in 1108 bytes} [Fri Jul 1 16:28:32 2022] GET /identity/v3/domains/default => generated 199 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:28:32.164413 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f257f4b-56ef-4386-b9a9-ec99c830c200 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:28:32.167064 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 515/1021] 10.222.0.57 () {58 vars in 1023 bytes} [Fri Jul 1 16:28:32 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 01 16:28:32.197327 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1820b0ca-9a7c-4697-8afc-fe4d35810633 None None] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:28:32.199534 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:28:32.277859 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:28:32.285700 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f257f4b-56ef-4386-b9a9-ec99c830c200 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:28:32.321587 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf7fbc9-bd29-4422-a70e-06bb70cdb8a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:28:32.322250 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf7fbc9-bd29-4422-a70e-06bb70cdb8a0 None admin] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:28:32.322566 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf7fbc9-bd29-4422-a70e-06bb70cdb8a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:28:32.322841 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf7fbc9-bd29-4422-a70e-06bb70cdb8a0 None admin] PATH_INFO: `/v3/projects` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:28:32.323548 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf7fbc9-bd29-4422-a70e-06bb70cdb8a0 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:28:32.324971 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf7fbc9-bd29-4422-a70e-06bb70cdb8a0 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:28:32.325478 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1820b0ca-9a7c-4697-8afc-fe4d35810633 None None] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:28:32.353264 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 507/1022] 10.222.0.57 () {60 vars in 1141 bytes} [Fri Jul 1 16:28:32 2022] POST /identity/v3/projects => generated 311 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 16:28:32.377911 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b71eb1e-6587-4605-8b77-d881930847ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:28:32.379774 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b71eb1e-6587-4605-8b77-d881930847ec None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:28:32.380064 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b71eb1e-6587-4605-8b77-d881930847ec None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:28:32.380345 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b71eb1e-6587-4605-8b77-d881930847ec None admin] PATH_INFO: `/v3/services/image` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:28:32.381039 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b71eb1e-6587-4605-8b77-d881930847ec None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:28:32.382561 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b71eb1e-6587-4605-8b77-d881930847ec None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:28:32.395831 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application [None req-0b71eb1e-6587-4605-8b77-d881930847ec None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 01 16:28:32.395831 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 16:28:32.395831 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 01 16:28:32.395831 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 01 16:28:32.395831 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 01 16:28:32.395831 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 01 16:28:32.395831 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 01 16:28:32.395831 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 01 16:28:32.395831 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 01 16:28:32.395831 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application Jul 01 16:28:32.395831 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 01 16:28:32.395831 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application Jul 01 16:28:32.395831 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 16:28:32.395831 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 16:28:32.395831 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 16:28:32.395831 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 16:28:32.395831 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 16:28:32.396910 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 16:28:32.396910 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 16:28:32.396910 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 16:28:32.396910 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 16:28:32.396910 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 16:28:32.396910 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 16:28:32.396910 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 01 16:28:32.396910 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 01 16:28:32.396910 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 01 16:28:32.396910 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 01 16:28:32.396910 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 16:28:32.396910 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 16:28:32.396910 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 16:28:32.396910 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 16:28:32.396910 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 16:28:32.396910 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 16:28:32.397835 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 16:28:32.397835 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application with Lock( Jul 01 16:28:32.397835 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 16:28:32.397835 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self._enter() Jul 01 16:28:32.397835 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 16:28:32.397835 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 16:28:32.397835 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 16:28:32.397835 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.creator() Jul 01 16:28:32.397835 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 16:28:32.397835 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application created_value = creator( Jul 01 16:28:32.397835 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 01 16:28:32.397835 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 01 16:28:32.397835 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 01 16:28:32.397835 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application Jul 01 16:28:32.397835 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 516/1023] 10.222.0.57 () {56 vars in 1105 bytes} [Fri Jul 1 16:28:32 2022] GET /identity/v3/services/image => generated 86 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 01 16:28:32.404628 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf7fbc9-bd29-4422-a70e-06bb70cdb8a0 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:28:32.486297 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6599eff5-a490-4ccb-9b77-044dfe45e484 None None] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:28:32.486655 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6599eff5-a490-4ccb-9b77-044dfe45e484 None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:28:32.486884 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6599eff5-a490-4ccb-9b77-044dfe45e484 None None] PATH_INFO: `/` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:28:32.489098 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 517/1024] 10.222.0.57 () {54 vars in 926 bytes} [Fri Jul 1 16:28:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 01 16:28:32.494650 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161cb014-84d7-441c-a050-d0da79a97cdf None None] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:28:32.494946 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161cb014-84d7-441c-a050-d0da79a97cdf None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:28:32.495230 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161cb014-84d7-441c-a050-d0da79a97cdf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:28:32.512521 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.auth.core [None req-161cb014-84d7-441c-a050-d0da79a97cdf None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80312) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:28:32.513869 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-161cb014-84d7-441c-a050-d0da79a97cdf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:28:32.527258 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf7fbc9-bd29-4422-a70e-06bb70cdb8a0 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:28:32.571087 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429cd3bb-a578-47d3-ab2a-8fd2e9b151e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:28:32.571779 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429cd3bb-a578-47d3-ab2a-8fd2e9b151e1 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:28:32.572124 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429cd3bb-a578-47d3-ab2a-8fd2e9b151e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:28:32.572425 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429cd3bb-a578-47d3-ab2a-8fd2e9b151e1 None admin] PATH_INFO: `/v3/services` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:28:32.573156 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-429cd3bb-a578-47d3-ab2a-8fd2e9b151e1 None admin] RBAC: Adding query filter params (name=image) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:28:32.573469 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-429cd3bb-a578-47d3-ab2a-8fd2e9b151e1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:28:32.574934 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-429cd3bb-a578-47d3-ab2a-8fd2e9b151e1 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:28:32.580922 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 508/1025] 10.222.0.57 () {56 vars in 1108 bytes} [Fri Jul 1 16:28:32 2022] GET /identity/v3/services?name=image => generated 122 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:28:32.587874 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429cd3bb-a578-47d3-ab2a-8fd2e9b151e1 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:28:32.595396 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=100 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:28:32.606069 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 518/1026] 10.222.0.57 () {58 vars in 1023 bytes} [Fri Jul 1 16:28:32 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 01 16:28:32.606505 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:28:32.633681 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161cb014-84d7-441c-a050-d0da79a97cdf None None] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:28:32.712770 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429cd3bb-a578-47d3-ab2a-8fd2e9b151e1 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:28:32.749143 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69535696-f3a3-4b63-97bb-c28168265393 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:28:32.749813 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69535696-f3a3-4b63-97bb-c28168265393 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:28:32.750099 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69535696-f3a3-4b63-97bb-c28168265393 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:28:32.750396 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69535696-f3a3-4b63-97bb-c28168265393 None admin] PATH_INFO: `/v3/services` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:28:32.751112 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69535696-f3a3-4b63-97bb-c28168265393 None admin] RBAC: Adding query filter params (type=image) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:28:32.751425 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69535696-f3a3-4b63-97bb-c28168265393 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:28:32.751629 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161cb014-84d7-441c-a050-d0da79a97cdf None None] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:28:32.752799 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69535696-f3a3-4b63-97bb-c28168265393 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:28:32.763559 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 509/1027] 10.222.0.57 () {56 vars in 1108 bytes} [Fri Jul 1 16:28:32 2022] GET /identity/v3/services?type=image => generated 350 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:28:32.765653 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69535696-f3a3-4b63-97bb-c28168265393 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:28:32.792837 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f33ef4-338c-43f4-9125-fe38365c02d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:28:32.793751 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f33ef4-338c-43f4-9125-fe38365c02d2 None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:28:32.794115 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f33ef4-338c-43f4-9125-fe38365c02d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:28:32.794608 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f33ef4-338c-43f4-9125-fe38365c02d2 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:28:32.807635 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f33ef4-338c-43f4-9125-fe38365c02d2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 01 16:28:32.817406 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application [None req-49f33ef4-338c-43f4-9125-fe38365c02d2 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 01 16:28:32.817406 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 16:28:32.817406 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 16:28:32.817406 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 16:28:32.817406 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 16:28:32.817406 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 16:28:32.817406 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 16:28:32.817406 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 16:28:32.817406 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 16:28:32.817406 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 16:28:32.817406 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 16:28:32.817406 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 16:28:32.817406 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 01 16:28:32.817406 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 01 16:28:32.817406 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 01 16:28:32.817406 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 01 16:28:32.818666 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 01 16:28:32.818666 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 01 16:28:32.818666 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 01 16:28:32.818666 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 01 16:28:32.818666 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 16:28:32.818666 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 16:28:32.818666 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 16:28:32.818666 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 16:28:32.818666 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 16:28:32.818666 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 16:28:32.818666 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 16:28:32.818666 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application with Lock( Jul 01 16:28:32.818666 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 16:28:32.818666 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self._enter() Jul 01 16:28:32.818666 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 16:28:32.818666 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 16:28:32.819817 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 16:28:32.819817 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.creator() Jul 01 16:28:32.819817 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 16:28:32.819817 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application created_value = creator( Jul 01 16:28:32.819817 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 01 16:28:32.819817 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 01 16:28:32.819817 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 01 16:28:32.819817 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 01 16:28:32.819817 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 01 16:28:32.819817 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 01 16:28:32.819817 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 01 16:28:32.819817 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application Jul 01 16:28:32.819817 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 519/1028] 10.222.0.57 () {56 vars in 1120 bytes} [Fri Jul 1 16:28:32 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 01 16:28:32.825013 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f33ef4-338c-43f4-9125-fe38365c02d2 None admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:28:32.853950 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:28:32.881779 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69535696-f3a3-4b63-97bb-c28168265393 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:28:32.895657 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=100 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:28:32.919939 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69dcde72-e4f3-4f49-8627-175769c2a2d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:28:32.920694 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69dcde72-e4f3-4f49-8627-175769c2a2d7 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:28:32.920895 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69dcde72-e4f3-4f49-8627-175769c2a2d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:28:32.921160 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69dcde72-e4f3-4f49-8627-175769c2a2d7 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:28:32.921956 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69dcde72-e4f3-4f49-8627-175769c2a2d7 None admin] RBAC: Adding query filter params (interface=public, service_id=8847b415cd164dca813cf1363978ad2f, region_id=RegionOne) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:28:32.922273 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69dcde72-e4f3-4f49-8627-175769c2a2d7 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:28:32.923681 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69dcde72-e4f3-4f49-8627-175769c2a2d7 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:28:32.930314 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 510/1029] 10.222.0.57 () {56 vars in 1251 bytes} [Fri Jul 1 16:28:32 2022] GET /identity/v3/endpoints?service_id=8847b415cd164dca813cf1363978ad2f&interface=public®ion_id=RegionOne => generated 194 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 01 16:28:32.935806 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69dcde72-e4f3-4f49-8627-175769c2a2d7 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:28:32.951469 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f33ef4-338c-43f4-9125-fe38365c02d2 None admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:28:32.992109 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d3babd-0fb1-4d20-9f2a-c7847418b7dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:28:32.993066 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d3babd-0fb1-4d20-9f2a-c7847418b7dc None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:28:32.993434 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d3babd-0fb1-4d20-9f2a-c7847418b7dc None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:28:32.993809 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d3babd-0fb1-4d20-9f2a-c7847418b7dc None admin] PATH_INFO: `/v3/roles` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:28:32.994904 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d3babd-0fb1-4d20-9f2a-c7847418b7dc None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80312) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:28:32.995457 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d3babd-0fb1-4d20-9f2a-c7847418b7dc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:28:32.997107 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d3babd-0fb1-4d20-9f2a-c7847418b7dc None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:28:33.006574 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 520/1030] 10.222.0.57 () {56 vars in 1115 bytes} [Fri Jul 1 16:28:32 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 01 16:28:33.012095 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d3babd-0fb1-4d20-9f2a-c7847418b7dc None admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:28:33.058890 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69dcde72-e4f3-4f49-8627-175769c2a2d7 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:28:33.102819 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c7de9c-776f-49c9-90f3-858cd9ce8e24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:28:33.103504 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c7de9c-776f-49c9-90f3-858cd9ce8e24 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:28:33.103782 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c7de9c-776f-49c9-90f3-858cd9ce8e24 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:28:33.104056 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c7de9c-776f-49c9-90f3-858cd9ce8e24 None admin] PATH_INFO: `/v3/services` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:28:33.104744 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c7de9c-776f-49c9-90f3-858cd9ce8e24 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:28:33.106115 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c7de9c-776f-49c9-90f3-858cd9ce8e24 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:28:33.114390 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 511/1031] 10.222.0.57 () {56 vars in 1087 bytes} [Fri Jul 1 16:28:32 2022] GET /identity/v3/services => generated 1957 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 01 16:28:33.121200 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d3babd-0fb1-4d20-9f2a-c7847418b7dc None admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:28:33.142572 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=100 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:28:33.162437 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d68f5e0-eda3-4f7a-9a0d-d60755f690dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:28:33.163122 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d68f5e0-eda3-4f7a-9a0d-d60755f690dd None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:28:33.163421 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d68f5e0-eda3-4f7a-9a0d-d60755f690dd None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:28:33.163679 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d68f5e0-eda3-4f7a-9a0d-d60755f690dd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:28:33.177751 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d68f5e0-eda3-4f7a-9a0d-d60755f690dd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:28:33.179252 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d68f5e0-eda3-4f7a-9a0d-d60755f690dd None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:28:33.198767 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application [None req-9d68f5e0-eda3-4f7a-9a0d-d60755f690dd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 16:28:33.198767 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 16:28:33.198767 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 16:28:33.198767 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 16:28:33.198767 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 16:28:33.198767 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 16:28:33.198767 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 16:28:33.198767 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 16:28:33.198767 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 16:28:33.198767 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 16:28:33.198767 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 16:28:33.198767 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 16:28:33.198767 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 01 16:28:33.198767 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 01 16:28:33.198767 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 01 16:28:33.199764 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 01 16:28:33.199764 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 16:28:33.199764 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 16:28:33.199764 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 16:28:33.199764 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 16:28:33.199764 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 16:28:33.199764 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 16:28:33.199764 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 16:28:33.199764 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application with Lock( Jul 01 16:28:33.199764 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 16:28:33.199764 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self._enter() Jul 01 16:28:33.199764 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 16:28:33.199764 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 16:28:33.199764 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 16:28:33.199764 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.creator() Jul 01 16:28:33.199764 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 16:28:33.199764 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application created_value = creator( Jul 01 16:28:33.200697 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 01 16:28:33.200697 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 01 16:28:33.200697 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 16:28:33.200697 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application Jul 01 16:28:33.200697 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 521/1032] 10.222.0.57 () {56 vars in 1108 bytes} [Fri Jul 1 16:28:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 16:28:33.207460 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c7de9c-776f-49c9-90f3-858cd9ce8e24 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:28:33.210744 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:28:33.333073 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c7de9c-776f-49c9-90f3-858cd9ce8e24 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:28:33.367051 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473190f4-4be1-4896-9675-edd61b6416fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:28:33.367711 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473190f4-4be1-4896-9675-edd61b6416fe None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:28:33.367993 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473190f4-4be1-4896-9675-edd61b6416fe None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:28:33.368267 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473190f4-4be1-4896-9675-edd61b6416fe None admin] PATH_INFO: `/v3/domains` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:28:33.368983 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473190f4-4be1-4896-9675-edd61b6416fe None admin] RBAC: Adding query filter params (name=Default) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:28:33.369286 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473190f4-4be1-4896-9675-edd61b6416fe None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:28:33.370663 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473190f4-4be1-4896-9675-edd61b6416fe None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:28:33.384196 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 512/1033] 10.222.0.57 () {56 vars in 1109 bytes} [Fri Jul 1 16:28:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:28:33.389650 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d68f5e0-eda3-4f7a-9a0d-d60755f690dd None admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:28:33.500628 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d68f5e0-eda3-4f7a-9a0d-d60755f690dd None admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:28:33.536958 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408cbf60-5ca1-494c-9cff-ed3779189b54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:28:33.537806 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408cbf60-5ca1-494c-9cff-ed3779189b54 None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:28:33.538104 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408cbf60-5ca1-494c-9cff-ed3779189b54 None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:28:33.538470 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408cbf60-5ca1-494c-9cff-ed3779189b54 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:28:33.547240 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-408cbf60-5ca1-494c-9cff-ed3779189b54 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:28:33.548744 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-408cbf60-5ca1-494c-9cff-ed3779189b54 None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:28:33.565570 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application [None req-408cbf60-5ca1-494c-9cff-ed3779189b54 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 01 16:28:33.565570 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 16:28:33.565570 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 16:28:33.565570 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 16:28:33.565570 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 16:28:33.565570 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 16:28:33.565570 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 16:28:33.565570 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 16:28:33.565570 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 16:28:33.565570 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 16:28:33.565570 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 16:28:33.565570 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 16:28:33.565570 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 01 16:28:33.565570 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 01 16:28:33.565570 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 01 16:28:33.565570 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 01 16:28:33.575166 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 16:28:33.575166 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 16:28:33.575166 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 01 16:28:33.575166 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 01 16:28:33.575166 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 01 16:28:33.575166 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 01 16:28:33.575166 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 16:28:33.575166 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 16:28:33.575166 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 16:28:33.575166 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 16:28:33.575166 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 16:28:33.575166 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application with Lock( Jul 01 16:28:33.575166 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 16:28:33.575166 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self._enter() Jul 01 16:28:33.575166 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 16:28:33.575166 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 16:28:33.575166 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 16:28:33.575166 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.creator() Jul 01 16:28:33.576112 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 16:28:33.576112 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application created_value = creator( Jul 01 16:28:33.576112 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 01 16:28:33.576112 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 01 16:28:33.576112 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 01 16:28:33.576112 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 01 16:28:33.576112 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 01 16:28:33.576112 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 01 16:28:33.576112 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 01 16:28:33.576112 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application Jul 01 16:28:33.576112 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 522/1034] 10.222.0.57 () {56 vars in 1093 bytes} [Fri Jul 1 16:28:33 2022] GET /identity/v3/users/nova => generated 82 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 01 16:28:33.579285 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473190f4-4be1-4896-9675-edd61b6416fe None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:28:33.650476 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:28:33.701209 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473190f4-4be1-4896-9675-edd61b6416fe None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:28:33.737565 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8577a24d-b63c-42f1-887b-8207b72aa9fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:28:33.738815 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8577a24d-b63c-42f1-887b-8207b72aa9fd None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:28:33.739105 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8577a24d-b63c-42f1-887b-8207b72aa9fd None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:28:33.739385 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8577a24d-b63c-42f1-887b-8207b72aa9fd None admin] PATH_INFO: `/v3/users` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:28:33.740234 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8577a24d-b63c-42f1-887b-8207b72aa9fd None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:28:33.740547 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8577a24d-b63c-42f1-887b-8207b72aa9fd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:28:33.741977 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8577a24d-b63c-42f1-887b-8207b72aa9fd None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:28:33.764724 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 513/1035] 10.222.0.57 () {56 vars in 1133 bytes} [Fri Jul 1 16:28:33 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:28:33.766041 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408cbf60-5ca1-494c-9cff-ed3779189b54 None admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:28:33.797920 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=100 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:28:33.870949 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408cbf60-5ca1-494c-9cff-ed3779189b54 None admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:28:33.904936 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45521b4e-b1c5-4356-8a88-a76548aa5666 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:28:33.905709 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45521b4e-b1c5-4356-8a88-a76548aa5666 None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:28:33.905900 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45521b4e-b1c5-4356-8a88-a76548aa5666 None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:28:33.906142 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45521b4e-b1c5-4356-8a88-a76548aa5666 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:28:33.975033 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45521b4e-b1c5-4356-8a88-a76548aa5666 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:28:33.976479 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45521b4e-b1c5-4356-8a88-a76548aa5666 None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:28:34.005238 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=100 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:28:34.091433 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 523/1036] 10.222.0.57 () {58 vars in 1282 bytes} [Fri Jul 1 16:28:33 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 01 16:28:34.097696 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8577a24d-b63c-42f1-887b-8207b72aa9fd None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:28:34.217255 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8577a24d-b63c-42f1-887b-8207b72aa9fd None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:28:34.239018 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:28:34.264417 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7beee87e-fef3-4c30-8f2c-20772605a72f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:28:34.265407 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7beee87e-fef3-4c30-8f2c-20772605a72f None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:28:34.265690 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7beee87e-fef3-4c30-8f2c-20772605a72f None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:28:34.266145 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7beee87e-fef3-4c30-8f2c-20772605a72f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:28:34.279471 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7beee87e-fef3-4c30-8f2c-20772605a72f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:28:34.280936 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7beee87e-fef3-4c30-8f2c-20772605a72f None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:28:34.303319 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application [None req-7beee87e-fef3-4c30-8f2c-20772605a72f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 16:28:34.303319 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 16:28:34.303319 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 16:28:34.303319 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 16:28:34.303319 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 16:28:34.303319 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 16:28:34.303319 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 16:28:34.303319 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 16:28:34.303319 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 16:28:34.303319 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 16:28:34.303319 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 16:28:34.303319 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 16:28:34.303319 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 01 16:28:34.303319 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 01 16:28:34.303319 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 01 16:28:34.304299 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 01 16:28:34.304299 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 16:28:34.304299 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 16:28:34.304299 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 16:28:34.304299 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 16:28:34.304299 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 16:28:34.304299 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 16:28:34.304299 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 16:28:34.304299 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application with Lock( Jul 01 16:28:34.304299 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 16:28:34.304299 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._enter() Jul 01 16:28:34.304299 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 16:28:34.304299 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 16:28:34.304299 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 16:28:34.304299 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.creator() Jul 01 16:28:34.304299 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 16:28:34.304299 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application created_value = creator( Jul 01 16:28:34.311089 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 01 16:28:34.311089 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 01 16:28:34.311089 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 16:28:34.311089 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Jul 01 16:28:34.311089 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 514/1037] 10.222.0.57 () {56 vars in 1108 bytes} [Fri Jul 1 16:28:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 01 16:28:34.313081 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45521b4e-b1c5-4356-8a88-a76548aa5666 None admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:28:34.436514 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45521b4e-b1c5-4356-8a88-a76548aa5666 None admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:28:34.466291 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=100 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:28:34.473574 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00e5c42-697a-49c2-b7b0-360accfdc166 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:28:34.474252 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00e5c42-697a-49c2-b7b0-360accfdc166 None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:28:34.474573 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00e5c42-697a-49c2-b7b0-360accfdc166 None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:28:34.474855 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00e5c42-697a-49c2-b7b0-360accfdc166 None admin] PATH_INFO: `/v3/domains` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:28:34.475591 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00e5c42-697a-49c2-b7b0-360accfdc166 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80312) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:28:34.475904 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00e5c42-697a-49c2-b7b0-360accfdc166 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:28:34.477288 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00e5c42-697a-49c2-b7b0-360accfdc166 None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:28:34.490681 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 524/1038] 10.222.0.57 () {56 vars in 1109 bytes} [Fri Jul 1 16:28:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:28:34.494981 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7beee87e-fef3-4c30-8f2c-20772605a72f None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:28:34.511243 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b1b75c-d45d-4edf-ab56-135cae4ba195 None None] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:28:34.511243 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b1b75c-d45d-4edf-ab56-135cae4ba195 None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:28:34.511243 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b1b75c-d45d-4edf-ab56-135cae4ba195 None None] PATH_INFO: `/` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:28:34.512597 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 525/1039] 10.222.0.57 () {54 vars in 926 bytes} [Fri Jul 1 16:28:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 01 16:28:34.517130 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6883feb-61f2-4f9c-a8f5-2a1a83e236b0 None None] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:28:34.517424 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6883feb-61f2-4f9c-a8f5-2a1a83e236b0 None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:28:34.517713 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6883feb-61f2-4f9c-a8f5-2a1a83e236b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:28:34.535567 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.auth.core [None req-e6883feb-61f2-4f9c-a8f5-2a1a83e236b0 None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80312) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:28:34.536765 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-e6883feb-61f2-4f9c-a8f5-2a1a83e236b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:28:34.608859 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7beee87e-fef3-4c30-8f2c-20772605a72f None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:28:34.610410 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 526/1040] 10.222.0.57 () {58 vars in 1023 bytes} [Fri Jul 1 16:28:34 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 01 16:28:34.636172 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6883feb-61f2-4f9c-a8f5-2a1a83e236b0 None None] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:28:34.653396 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fce9888-e6d8-47f2-a9ae-40af12855735 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:28:34.654197 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fce9888-e6d8-47f2-a9ae-40af12855735 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:28:34.654520 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fce9888-e6d8-47f2-a9ae-40af12855735 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:28:34.654802 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fce9888-e6d8-47f2-a9ae-40af12855735 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:28:34.661940 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fce9888-e6d8-47f2-a9ae-40af12855735 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:28:34.663420 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fce9888-e6d8-47f2-a9ae-40af12855735 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:28:34.678047 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application [None req-6fce9888-e6d8-47f2-a9ae-40af12855735 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 01 16:28:34.678047 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 16:28:34.678047 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 16:28:34.678047 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 16:28:34.678047 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 16:28:34.678047 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 16:28:34.678047 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 16:28:34.678047 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 16:28:34.678047 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 16:28:34.678047 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 16:28:34.678047 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 16:28:34.678047 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 16:28:34.678047 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 01 16:28:34.678047 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 01 16:28:34.678047 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 01 16:28:34.679021 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 01 16:28:34.679021 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 16:28:34.679021 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 16:28:34.679021 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 16:28:34.679021 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 16:28:34.679021 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 16:28:34.679021 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 16:28:34.679021 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 16:28:34.679021 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application with Lock( Jul 01 16:28:34.679021 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 16:28:34.679021 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._enter() Jul 01 16:28:34.679021 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 16:28:34.679021 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 16:28:34.679021 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 16:28:34.679021 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.creator() Jul 01 16:28:34.679021 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 16:28:34.679021 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application created_value = creator( Jul 01 16:28:34.680160 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 01 16:28:34.680160 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 01 16:28:34.680160 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 01 16:28:34.680160 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 01 16:28:34.680160 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 01 16:28:34.680160 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 01 16:28:34.680160 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 01 16:28:34.680160 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Jul 01 16:28:34.680160 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 515/1041] 10.222.0.57 () {56 vars in 1111 bytes} [Fri Jul 1 16:28:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 16:28:34.684984 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fce9888-e6d8-47f2-a9ae-40af12855735 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:28:34.720728 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:28:34.759136 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6883feb-61f2-4f9c-a8f5-2a1a83e236b0 None None] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:28:34.779979 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=100 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:28:34.797488 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b644061-d27e-4c3f-8b89-fb569c46314e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:28:34.798283 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b644061-d27e-4c3f-8b89-fb569c46314e None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:28:34.798595 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b644061-d27e-4c3f-8b89-fb569c46314e None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:28:34.798854 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fce9888-e6d8-47f2-a9ae-40af12855735 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:28:34.798957 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b644061-d27e-4c3f-8b89-fb569c46314e None admin] PATH_INFO: `/v3/domains/default` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:28:34.800143 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b644061-d27e-4c3f-8b89-fb569c46314e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:28:34.802063 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b644061-d27e-4c3f-8b89-fb569c46314e None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:28:34.804070 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 527/1042] 10.222.0.57 () {56 vars in 1108 bytes} [Fri Jul 1 16:28:34 2022] GET /identity/v3/domains/default => generated 199 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 01 16:28:34.809374 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b644061-d27e-4c3f-8b89-fb569c46314e None admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:28:34.840639 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a209ef-3868-4b00-9b28-eae3c887dcc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:28:34.841400 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a209ef-3868-4b00-9b28-eae3c887dcc4 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:28:34.841682 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a209ef-3868-4b00-9b28-eae3c887dcc4 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:28:34.842247 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a209ef-3868-4b00-9b28-eae3c887dcc4 None admin] PATH_INFO: `/v3/projects` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:28:34.843228 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a209ef-3868-4b00-9b28-eae3c887dcc4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:28:34.843542 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a209ef-3868-4b00-9b28-eae3c887dcc4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:28:34.844977 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a209ef-3868-4b00-9b28-eae3c887dcc4 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:28:34.858413 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 516/1043] 10.222.0.57 () {56 vars in 1148 bytes} [Fri Jul 1 16:28:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 01 16:28:34.864427 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a209ef-3868-4b00-9b28-eae3c887dcc4 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:28:34.926726 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b644061-d27e-4c3f-8b89-fb569c46314e None admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:28:34.964864 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba37a555-9170-41b2-a067-0298ea129e3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:28:34.965702 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba37a555-9170-41b2-a067-0298ea129e3f None admin] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:28:34.966004 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba37a555-9170-41b2-a067-0298ea129e3f None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:28:34.966624 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba37a555-9170-41b2-a067-0298ea129e3f None admin] PATH_INFO: `/v3/users` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:28:34.967096 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba37a555-9170-41b2-a067-0298ea129e3f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:28:34.968551 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba37a555-9170-41b2-a067-0298ea129e3f None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:28:34.973629 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:28:34.984135 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a209ef-3868-4b00-9b28-eae3c887dcc4 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:28:35.002378 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 528/1044] 10.222.0.57 () {60 vars in 1132 bytes} [Fri Jul 1 16:28:34 2022] POST /identity/v3/users => generated 285 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 01 16:28:35.042972 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c6bf74-eb63-4c9b-b9ca-8cb05dabc385 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:28:35.043799 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c6bf74-eb63-4c9b-b9ca-8cb05dabc385 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:28:35.044078 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c6bf74-eb63-4c9b-b9ca-8cb05dabc385 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:28:35.044355 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c6bf74-eb63-4c9b-b9ca-8cb05dabc385 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:28:35.045203 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c6bf74-eb63-4c9b-b9ca-8cb05dabc385 None admin] RBAC: Adding query filter params (role.id=c3a62bbbb749474aad6655e693c8963d, scope.project.id=e0788e6725a84e57a4174284c1845755, user.id=7a50f745b31941b7a7265c0fc852f48e) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:28:35.045514 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c6bf74-eb63-4c9b-b9ca-8cb05dabc385 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:28:35.047028 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c6bf74-eb63-4c9b-b9ca-8cb05dabc385 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:28:35.058381 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 517/1045] 10.222.0.57 () {56 vars in 1374 bytes} [Fri Jul 1 16:28:34 2022] GET /identity/v3/role_assignments?user.id=7a50f745b31941b7a7265c0fc852f48e&scope.project.id=e0788e6725a84e57a4174284c1845755&role.id=c3a62bbbb749474aad6655e693c8963d => generated 259 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:28:35.284811 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9e133d-c597-4a0a-8ddb-78c24a2b109f None None] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:28:35.285324 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9e133d-c597-4a0a-8ddb-78c24a2b109f None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:28:35.285622 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9e133d-c597-4a0a-8ddb-78c24a2b109f None None] PATH_INFO: `/` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:28:35.287200 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 529/1046] 10.222.0.57 () {54 vars in 926 bytes} [Fri Jul 1 16:28:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 16:28:35.292256 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c3387a-4cf3-4d17-860f-43436c98e6f0 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:28:35.292707 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c3387a-4cf3-4d17-860f-43436c98e6f0 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:28:35.292873 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c3387a-4cf3-4d17-860f-43436c98e6f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:28:35.311252 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-94c3387a-4cf3-4d17-860f-43436c98e6f0 None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:28:35.312443 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-94c3387a-4cf3-4d17-860f-43436c98e6f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:28:35.383977 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 518/1047] 10.222.0.57 () {58 vars in 1023 bytes} [Fri Jul 1 16:28:35 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 01 16:28:35.407327 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9e133d-c597-4a0a-8ddb-78c24a2b109f None None] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:28:35.457629 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:28:35.521180 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9e133d-c597-4a0a-8ddb-78c24a2b109f None None] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:28:35.557517 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6933200-b747-4130-8140-4630f2d01883 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:28:35.558229 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6933200-b747-4130-8140-4630f2d01883 None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:28:35.558542 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6933200-b747-4130-8140-4630f2d01883 None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:28:35.558828 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6933200-b747-4130-8140-4630f2d01883 None admin] PATH_INFO: `/v3/services/image` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:28:35.559485 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6933200-b747-4130-8140-4630f2d01883 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:28:35.560852 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6933200-b747-4130-8140-4630f2d01883 None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:28:35.572982 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application [None req-e6933200-b747-4130-8140-4630f2d01883 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 01 16:28:35.572982 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 16:28:35.572982 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 01 16:28:35.572982 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 01 16:28:35.572982 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 01 16:28:35.572982 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 01 16:28:35.572982 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 01 16:28:35.572982 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 01 16:28:35.572982 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 01 16:28:35.572982 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application Jul 01 16:28:35.572982 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 01 16:28:35.572982 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application Jul 01 16:28:35.572982 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 16:28:35.572982 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 16:28:35.572982 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 16:28:35.572982 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 16:28:35.572982 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 16:28:35.574086 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 16:28:35.574086 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 16:28:35.574086 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 16:28:35.574086 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 16:28:35.574086 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 16:28:35.574086 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 16:28:35.574086 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 01 16:28:35.574086 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 01 16:28:35.574086 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 01 16:28:35.574086 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 01 16:28:35.574086 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 16:28:35.574086 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 16:28:35.574086 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 16:28:35.574086 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 16:28:35.574086 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 16:28:35.574086 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 16:28:35.574974 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 16:28:35.574974 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application with Lock( Jul 01 16:28:35.574974 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 16:28:35.574974 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self._enter() Jul 01 16:28:35.574974 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 16:28:35.574974 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 16:28:35.574974 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 16:28:35.574974 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.creator() Jul 01 16:28:35.574974 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 16:28:35.574974 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application created_value = creator( Jul 01 16:28:35.574974 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 01 16:28:35.574974 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 01 16:28:35.574974 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 01 16:28:35.574974 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application Jul 01 16:28:35.574974 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 530/1048] 10.222.0.57 () {56 vars in 1105 bytes} [Fri Jul 1 16:28:35 2022] GET /identity/v3/services/image => generated 86 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 16:28:35.582390 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c3387a-4cf3-4d17-860f-43436c98e6f0 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:28:35.666871 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:28:35.682680 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c3387a-4cf3-4d17-860f-43436c98e6f0 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:28:35.719186 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4558176d-b411-4c04-a833-10dac5dbeaef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:28:35.719859 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4558176d-b411-4c04-a833-10dac5dbeaef None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:28:35.720146 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4558176d-b411-4c04-a833-10dac5dbeaef None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:28:35.720420 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4558176d-b411-4c04-a833-10dac5dbeaef None admin] PATH_INFO: `/v3/services` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:28:35.721132 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4558176d-b411-4c04-a833-10dac5dbeaef None admin] RBAC: Adding query filter params (name=image) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:28:35.721442 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4558176d-b411-4c04-a833-10dac5dbeaef None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:28:35.723943 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4558176d-b411-4c04-a833-10dac5dbeaef None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:28:35.733680 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 519/1049] 10.222.0.57 () {56 vars in 1108 bytes} [Fri Jul 1 16:28:35 2022] GET /identity/v3/services?name=image => generated 122 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 01 16:28:35.736742 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6933200-b747-4130-8140-4630f2d01883 None admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:28:35.855528 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6933200-b747-4130-8140-4630f2d01883 None admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:28:35.865274 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=100 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:28:35.891780 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223ba586-623c-4292-8afa-197fcb09a369 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:28:35.892441 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223ba586-623c-4292-8afa-197fcb09a369 None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:28:35.892726 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223ba586-623c-4292-8afa-197fcb09a369 None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:28:35.893004 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223ba586-623c-4292-8afa-197fcb09a369 None admin] PATH_INFO: `/v3/services` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:28:35.894409 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-223ba586-623c-4292-8afa-197fcb09a369 None admin] RBAC: Adding query filter params (type=image) {{(pid=80312) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:28:35.894795 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-223ba586-623c-4292-8afa-197fcb09a369 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:28:35.896245 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-223ba586-623c-4292-8afa-197fcb09a369 None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:28:35.905956 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 531/1050] 10.222.0.57 () {56 vars in 1108 bytes} [Fri Jul 1 16:28:35 2022] GET /identity/v3/services?type=image => generated 350 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:28:35.907417 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4558176d-b411-4c04-a833-10dac5dbeaef None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:28:36.010772 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4558176d-b411-4c04-a833-10dac5dbeaef None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:28:36.045094 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62314a0f-7c9c-4c57-9540-df150f9f5fcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:28:36.045753 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62314a0f-7c9c-4c57-9540-df150f9f5fcd None admin] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:28:36.046038 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62314a0f-7c9c-4c57-9540-df150f9f5fcd None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:28:36.046317 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62314a0f-7c9c-4c57-9540-df150f9f5fcd None admin] PATH_INFO: `/v3/endpoints` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:28:36.047079 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62314a0f-7c9c-4c57-9540-df150f9f5fcd None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:28:36.048446 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62314a0f-7c9c-4c57-9540-df150f9f5fcd None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:28:36.072416 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 520/1051] 10.222.0.57 () {60 vars in 1144 bytes} [Fri Jul 1 16:28:35 2022] POST /identity/v3/endpoints => generated 327 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 01 16:28:37.117199 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478e9146-abdd-4caa-9690-5914ec4a4d42 None None] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:28:37.117695 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478e9146-abdd-4caa-9690-5914ec4a4d42 None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:28:37.117695 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478e9146-abdd-4caa-9690-5914ec4a4d42 None None] PATH_INFO: `/` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:28:37.119207 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 532/1052] 10.222.0.57 () {54 vars in 926 bytes} [Fri Jul 1 16:28:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 16:28:37.124181 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2ed8b8-3bc1-48f7-b756-dc15a42125b8 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:28:37.124563 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2ed8b8-3bc1-48f7-b756-dc15a42125b8 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:28:37.124727 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2ed8b8-3bc1-48f7-b756-dc15a42125b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:28:37.136945 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:28:37.139426 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb337d9-c3b0-44a2-a146-ba40542c4bcb None None] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:28:37.139717 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb337d9-c3b0-44a2-a146-ba40542c4bcb None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:28:37.140240 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb337d9-c3b0-44a2-a146-ba40542c4bcb None None] PATH_INFO: `/` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:28:37.141630 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 533/1053] 10.222.0.57 () {54 vars in 926 bytes} [Fri Jul 1 16:28:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 16:28:37.142452 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-ab2ed8b8-3bc1-48f7-b756-dc15a42125b8 None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:28:37.144714 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-ab2ed8b8-3bc1-48f7-b756-dc15a42125b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:28:37.147051 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d446d1-5dea-4c76-b55d-212da080e50c None None] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:28:37.147342 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d446d1-5dea-4c76-b55d-212da080e50c None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:28:37.147625 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d446d1-5dea-4c76-b55d-212da080e50c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:28:37.164961 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.auth.core [None req-39d446d1-5dea-4c76-b55d-212da080e50c None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80312) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:28:37.166144 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-39d446d1-5dea-4c76-b55d-212da080e50c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:28:37.233212 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 521/1054] 10.222.0.57 () {58 vars in 1023 bytes} [Fri Jul 1 16:28:37 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 01 16:28:37.247020 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 534/1055] 10.222.0.57 () {58 vars in 1023 bytes} [Fri Jul 1 16:28:37 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 01 16:28:37.256292 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2ed8b8-3bc1-48f7-b756-dc15a42125b8 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:28:37.274638 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d446d1-5dea-4c76-b55d-212da080e50c None None] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:28:37.364545 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2ed8b8-3bc1-48f7-b756-dc15a42125b8 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:28:37.378285 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d446d1-5dea-4c76-b55d-212da080e50c None None] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:28:37.380216 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=100 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:28:37.395925 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:28:37.401989 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70233890-9c22-436b-97eb-43a2422dae6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:28:37.403018 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70233890-9c22-436b-97eb-43a2422dae6c None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:28:37.403300 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70233890-9c22-436b-97eb-43a2422dae6c None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:28:37.403573 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70233890-9c22-436b-97eb-43a2422dae6c None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:28:37.414525 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70233890-9c22-436b-97eb-43a2422dae6c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 01 16:28:37.419932 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b587a2-cc1b-4016-a7cb-a6f8523134d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:28:37.420731 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b587a2-cc1b-4016-a7cb-a6f8523134d9 None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:28:37.421020 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b587a2-cc1b-4016-a7cb-a6f8523134d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:28:37.421299 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b587a2-cc1b-4016-a7cb-a6f8523134d9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:28:37.422571 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application [None req-70233890-9c22-436b-97eb-43a2422dae6c None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 01 16:28:37.422571 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 16:28:37.422571 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 16:28:37.422571 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 16:28:37.422571 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 16:28:37.422571 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 16:28:37.422571 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 16:28:37.422571 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 16:28:37.422571 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 16:28:37.422571 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 16:28:37.422571 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 16:28:37.422571 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 16:28:37.422571 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 01 16:28:37.422571 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 01 16:28:37.422571 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 01 16:28:37.422571 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 01 16:28:37.423518 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 01 16:28:37.423518 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 01 16:28:37.423518 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 01 16:28:37.423518 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 01 16:28:37.423518 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 16:28:37.423518 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 16:28:37.423518 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 16:28:37.423518 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 16:28:37.423518 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 16:28:37.423518 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 16:28:37.423518 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 16:28:37.423518 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application with Lock( Jul 01 16:28:37.423518 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 16:28:37.423518 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._enter() Jul 01 16:28:37.423518 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 16:28:37.423518 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 16:28:37.424368 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 16:28:37.424368 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.creator() Jul 01 16:28:37.424368 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 16:28:37.424368 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application created_value = creator( Jul 01 16:28:37.424368 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 01 16:28:37.424368 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 01 16:28:37.424368 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 01 16:28:37.424368 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 01 16:28:37.424368 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 01 16:28:37.424368 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 01 16:28:37.424368 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 01 16:28:37.424368 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Jul 01 16:28:37.424368 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 522/1056] 10.222.0.57 () {56 vars in 1120 bytes} [Fri Jul 1 16:28:37 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 16:28:37.430266 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70233890-9c22-436b-97eb-43a2422dae6c None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:28:37.435710 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b587a2-cc1b-4016-a7cb-a6f8523134d9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 01 16:28:37.444121 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application [None req-76b587a2-cc1b-4016-a7cb-a6f8523134d9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 01 16:28:37.444121 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 16:28:37.444121 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 16:28:37.444121 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 16:28:37.444121 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 16:28:37.444121 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 16:28:37.444121 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 16:28:37.444121 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 16:28:37.444121 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 16:28:37.444121 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 16:28:37.444121 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 16:28:37.444121 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 16:28:37.444121 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 01 16:28:37.444121 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 01 16:28:37.444121 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 01 16:28:37.444121 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 01 16:28:37.446702 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 01 16:28:37.446702 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 01 16:28:37.446702 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 01 16:28:37.446702 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 01 16:28:37.446702 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 16:28:37.446702 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 16:28:37.446702 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 16:28:37.446702 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 16:28:37.446702 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 16:28:37.446702 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 16:28:37.446702 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 16:28:37.446702 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application with Lock( Jul 01 16:28:37.446702 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 16:28:37.446702 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self._enter() Jul 01 16:28:37.446702 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 16:28:37.446702 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 16:28:37.447566 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 16:28:37.447566 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.creator() Jul 01 16:28:37.447566 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 16:28:37.447566 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application created_value = creator( Jul 01 16:28:37.447566 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 01 16:28:37.447566 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 01 16:28:37.447566 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 01 16:28:37.447566 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 01 16:28:37.447566 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 01 16:28:37.447566 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 01 16:28:37.447566 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 01 16:28:37.447566 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application Jul 01 16:28:37.447566 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 535/1057] 10.222.0.57 () {56 vars in 1096 bytes} [Fri Jul 1 16:28:37 2022] GET /identity/v3/roles/admin => generated 83 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 01 16:28:37.451193 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b587a2-cc1b-4016-a7cb-a6f8523134d9 None admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:28:37.540524 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70233890-9c22-436b-97eb-43a2422dae6c None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:28:37.562883 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b587a2-cc1b-4016-a7cb-a6f8523134d9 None admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:28:37.578454 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0f3091-1fad-472f-80f8-c69ab8bb9a00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:28:37.579241 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0f3091-1fad-472f-80f8-c69ab8bb9a00 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:28:37.579465 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0f3091-1fad-472f-80f8-c69ab8bb9a00 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:28:37.579750 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0f3091-1fad-472f-80f8-c69ab8bb9a00 None admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:28:37.580506 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd0f3091-1fad-472f-80f8-c69ab8bb9a00 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:28:37.580817 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd0f3091-1fad-472f-80f8-c69ab8bb9a00 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:28:37.582504 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd0f3091-1fad-472f-80f8-c69ab8bb9a00 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:28:37.593820 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 523/1058] 10.222.0.57 () {56 vars in 1115 bytes} [Fri Jul 1 16:28:37 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 01 16:28:37.596802 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0f3091-1fad-472f-80f8-c69ab8bb9a00 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:28:37.608234 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bcaaa7c-1b22-43d3-b4e9-ea6c681fc8d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:28:37.609545 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcaaa7c-1b22-43d3-b4e9-ea6c681fc8d5 None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:28:37.609836 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcaaa7c-1b22-43d3-b4e9-ea6c681fc8d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:28:37.610122 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcaaa7c-1b22-43d3-b4e9-ea6c681fc8d5 None admin] PATH_INFO: `/v3/roles` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:28:37.610919 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bcaaa7c-1b22-43d3-b4e9-ea6c681fc8d5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80312) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:28:37.611238 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bcaaa7c-1b22-43d3-b4e9-ea6c681fc8d5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:28:37.615573 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bcaaa7c-1b22-43d3-b4e9-ea6c681fc8d5 None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:28:37.626222 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 536/1059] 10.222.0.57 () {56 vars in 1099 bytes} [Fri Jul 1 16:28:37 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 01 16:28:37.632986 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bcaaa7c-1b22-43d3-b4e9-ea6c681fc8d5 None admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:28:37.659230 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:28:37.661397 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=100 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:28:37.712485 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0f3091-1fad-472f-80f8-c69ab8bb9a00 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:28:37.750615 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bcaaa7c-1b22-43d3-b4e9-ea6c681fc8d5 None admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:28:37.755221 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1da80e-4ece-4e15-b291-94ff426e4383 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:28:37.756311 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1da80e-4ece-4e15-b291-94ff426e4383 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:28:37.756738 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1da80e-4ece-4e15-b291-94ff426e4383 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:28:37.757156 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1da80e-4ece-4e15-b291-94ff426e4383 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:28:37.775412 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b1da80e-4ece-4e15-b291-94ff426e4383 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:28:37.778402 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b1da80e-4ece-4e15-b291-94ff426e4383 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:28:37.788319 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792e3d39-d84c-4acd-9eb5-c1b0997454d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:28:37.789214 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792e3d39-d84c-4acd-9eb5-c1b0997454d2 None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:28:37.789497 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792e3d39-d84c-4acd-9eb5-c1b0997454d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:28:37.789777 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792e3d39-d84c-4acd-9eb5-c1b0997454d2 None admin] PATH_INFO: `/v3/users/e0b10b53b2474b738cf28a04d0425c54` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:28:37.805236 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application [None req-8b1da80e-4ece-4e15-b291-94ff426e4383 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 16:28:37.805236 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 16:28:37.805236 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 16:28:37.805236 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 16:28:37.805236 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 16:28:37.805236 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 16:28:37.805236 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 16:28:37.805236 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 16:28:37.805236 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 16:28:37.805236 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 16:28:37.805236 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 16:28:37.805236 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 16:28:37.805236 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 01 16:28:37.805236 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 01 16:28:37.805236 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 01 16:28:37.806483 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 01 16:28:37.806483 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 16:28:37.806483 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 16:28:37.806483 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 16:28:37.806483 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 16:28:37.806483 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 16:28:37.806483 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 16:28:37.806483 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 16:28:37.806483 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application with Lock( Jul 01 16:28:37.806483 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 16:28:37.806483 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._enter() Jul 01 16:28:37.806483 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 16:28:37.806483 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 16:28:37.806483 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 16:28:37.806483 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.creator() Jul 01 16:28:37.806483 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 16:28:37.806483 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application created_value = creator( Jul 01 16:28:37.807711 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 01 16:28:37.807711 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 01 16:28:37.807711 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 16:28:37.807711 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Jul 01 16:28:37.807711 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 524/1060] 10.222.0.57 () {56 vars in 1108 bytes} [Fri Jul 1 16:28:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 01 16:28:37.811144 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-792e3d39-d84c-4acd-9eb5-c1b0997454d2 None admin] RBAC: Authorizing `identity:get_user(user_id=e0b10b53b2474b738cf28a04d0425c54)` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:28:37.812483 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1da80e-4ece-4e15-b291-94ff426e4383 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:28:37.812997 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-792e3d39-d84c-4acd-9eb5-c1b0997454d2 None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:28:37.819830 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 537/1061] 10.222.0.57 () {56 vars in 1177 bytes} [Fri Jul 1 16:28:37 2022] GET /identity/v3/users/e0b10b53b2474b738cf28a04d0425c54 => generated 285 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 01 16:28:37.820544 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792e3d39-d84c-4acd-9eb5-c1b0997454d2 None admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:28:37.936341 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=100 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:28:37.957671 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792e3d39-d84c-4acd-9eb5-c1b0997454d2 None admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:28:37.963327 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1da80e-4ece-4e15-b291-94ff426e4383 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:28:37.972874 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:28:38.003753 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba519e7c-404a-4cde-99fb-8f31901cd113 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:28:38.004706 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba519e7c-404a-4cde-99fb-8f31901cd113 None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:28:38.004980 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba519e7c-404a-4cde-99fb-8f31901cd113 None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:28:38.005259 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba519e7c-404a-4cde-99fb-8f31901cd113 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:28:38.007757 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80999eae-c700-4206-965f-25b700f048d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:28:38.008440 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80999eae-c700-4206-965f-25b700f048d2 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:28:38.008727 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80999eae-c700-4206-965f-25b700f048d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:28:38.009009 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80999eae-c700-4206-965f-25b700f048d2 None admin] PATH_INFO: `/v3/domains` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:28:38.009755 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80999eae-c700-4206-965f-25b700f048d2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:28:38.010097 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80999eae-c700-4206-965f-25b700f048d2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:28:38.011532 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80999eae-c700-4206-965f-25b700f048d2 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:28:38.025339 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 525/1062] 10.222.0.57 () {56 vars in 1109 bytes} [Fri Jul 1 16:28:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:28:38.030959 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80999eae-c700-4206-965f-25b700f048d2 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:28:38.089015 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba519e7c-404a-4cde-99fb-8f31901cd113 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:28:38.090989 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba519e7c-404a-4cde-99fb-8f31901cd113 None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:28:38.158837 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80999eae-c700-4206-965f-25b700f048d2 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:28:38.187507 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=100 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:28:38.197124 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13dd8a08-b96d-466b-bcab-2ba459198435 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:28:38.198209 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13dd8a08-b96d-466b-bcab-2ba459198435 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:28:38.198546 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13dd8a08-b96d-466b-bcab-2ba459198435 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:28:38.198814 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13dd8a08-b96d-466b-bcab-2ba459198435 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:28:38.208218 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13dd8a08-b96d-466b-bcab-2ba459198435 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:28:38.209671 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13dd8a08-b96d-466b-bcab-2ba459198435 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:28:38.217290 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 538/1063] 10.222.0.57 () {58 vars in 1282 bytes} [Fri Jul 1 16:28:37 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 397 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 01 16:28:38.218933 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1d97cf-c0c3-40c0-809c-723eb86620ac None None] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:28:38.219274 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1d97cf-c0c3-40c0-809c-723eb86620ac None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:28:38.219668 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1d97cf-c0c3-40c0-809c-723eb86620ac None None] PATH_INFO: `/` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:28:38.222509 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 539/1064] 10.222.0.57 () {54 vars in 926 bytes} [Fri Jul 1 16:28:38 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 16:28:38.224445 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1d97cf-c0c3-40c0-809c-723eb86620ac None None] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:28:38.224667 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application [None req-13dd8a08-b96d-466b-bcab-2ba459198435 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 01 16:28:38.224667 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 16:28:38.224667 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 16:28:38.224667 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 16:28:38.224667 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 16:28:38.224667 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 16:28:38.224667 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 16:28:38.224667 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 16:28:38.224667 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 16:28:38.224667 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 16:28:38.224667 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 16:28:38.224667 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 16:28:38.224667 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 01 16:28:38.224667 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 01 16:28:38.224667 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 01 16:28:38.224667 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 01 16:28:38.225565 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 16:28:38.225565 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 16:28:38.225565 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 01 16:28:38.225565 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 01 16:28:38.225565 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 01 16:28:38.225565 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 01 16:28:38.225565 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 16:28:38.225565 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 16:28:38.225565 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 16:28:38.225565 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 16:28:38.225565 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 16:28:38.225565 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application with Lock( Jul 01 16:28:38.225565 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 16:28:38.225565 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._enter() Jul 01 16:28:38.225565 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 16:28:38.225565 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 16:28:38.225565 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 16:28:38.225565 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.creator() Jul 01 16:28:38.226565 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 16:28:38.226565 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application created_value = creator( Jul 01 16:28:38.226565 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 01 16:28:38.226565 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 01 16:28:38.226565 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 01 16:28:38.226565 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 01 16:28:38.226565 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 01 16:28:38.226565 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 01 16:28:38.226565 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 01 16:28:38.226565 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Jul 01 16:28:38.226565 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 526/1065] 10.222.0.57 () {56 vars in 1093 bytes} [Fri Jul 1 16:28:38 2022] GET /identity/v3/users/nova => generated 82 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 01 16:28:38.230128 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa7f62e-446b-4e64-b617-7556a703b318 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:28:38.230128 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa7f62e-446b-4e64-b617-7556a703b318 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:28:38.230128 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa7f62e-446b-4e64-b617-7556a703b318 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:28:38.247171 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-daa7f62e-446b-4e64-b617-7556a703b318 None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:28:38.248381 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-daa7f62e-446b-4e64-b617-7556a703b318 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:28:38.269593 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:28:38.328722 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 527/1066] 10.222.0.57 () {58 vars in 1023 bytes} [Fri Jul 1 16:28:38 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 01 16:28:38.331395 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa7f62e-446b-4e64-b617-7556a703b318 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:28:38.343576 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1d97cf-c0c3-40c0-809c-723eb86620ac None None] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:28:38.386096 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff50800-b91a-42b5-9963-0758e4eb622a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:28:38.387042 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff50800-b91a-42b5-9963-0758e4eb622a None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:28:38.387327 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff50800-b91a-42b5-9963-0758e4eb622a None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:28:38.387607 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff50800-b91a-42b5-9963-0758e4eb622a None admin] PATH_INFO: `/v3/projects/08b05678e269471b80c2f94499c9d39a` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:28:38.388854 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff50800-b91a-42b5-9963-0758e4eb622a None admin] RBAC: Authorizing `identity:get_project(project_id=08b05678e269471b80c2f94499c9d39a)` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:28:38.390311 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff50800-b91a-42b5-9963-0758e4eb622a None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:28:38.392097 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 540/1067] 10.222.0.57 () {56 vars in 1186 bytes} [Fri Jul 1 16:28:38 2022] GET /identity/v3/projects/08b05678e269471b80c2f94499c9d39a => generated 311 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:28:38.395331 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff50800-b91a-42b5-9963-0758e4eb622a None admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:28:38.442332 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa7f62e-446b-4e64-b617-7556a703b318 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:28:38.450568 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=100 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:28:38.481694 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bea4ddb-d87d-48c5-83c7-f614e688f175 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:28:38.482391 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bea4ddb-d87d-48c5-83c7-f614e688f175 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:28:38.482680 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bea4ddb-d87d-48c5-83c7-f614e688f175 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:28:38.482950 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bea4ddb-d87d-48c5-83c7-f614e688f175 None admin] PATH_INFO: `/v3/users` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:28:38.483785 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bea4ddb-d87d-48c5-83c7-f614e688f175 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:28:38.484090 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bea4ddb-d87d-48c5-83c7-f614e688f175 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:28:38.485519 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bea4ddb-d87d-48c5-83c7-f614e688f175 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:28:38.503432 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff50800-b91a-42b5-9963-0758e4eb622a None admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:28:38.504781 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 528/1068] 10.222.0.57 () {56 vars in 1133 bytes} [Fri Jul 1 16:28:38 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:28:38.505637 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bea4ddb-d87d-48c5-83c7-f614e688f175 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:28:38.524444 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:28:38.546889 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bdae1d9-766e-4cc2-818d-081f4e6ae99d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:28:38.547550 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdae1d9-766e-4cc2-818d-081f4e6ae99d None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:28:38.547837 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdae1d9-766e-4cc2-818d-081f4e6ae99d None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:28:38.548122 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdae1d9-766e-4cc2-818d-081f4e6ae99d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:28:38.619522 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bea4ddb-d87d-48c5-83c7-f614e688f175 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:28:38.624028 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bdae1d9-766e-4cc2-818d-081f4e6ae99d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:28:38.625483 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bdae1d9-766e-4cc2-818d-081f4e6ae99d None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:28:38.663333 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776a9534-8f53-42d1-aee5-7390e4f73c92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:28:38.663995 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776a9534-8f53-42d1-aee5-7390e4f73c92 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:28:38.664279 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776a9534-8f53-42d1-aee5-7390e4f73c92 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:28:38.664558 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776a9534-8f53-42d1-aee5-7390e4f73c92 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:28:38.665402 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-776a9534-8f53-42d1-aee5-7390e4f73c92 None admin] RBAC: Adding query filter params (role.id=84ee739146934c1ab32c501ff8093644, scope.project.id=08b05678e269471b80c2f94499c9d39a, user.id=e0b10b53b2474b738cf28a04d0425c54) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:28:38.665719 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-776a9534-8f53-42d1-aee5-7390e4f73c92 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:28:38.667193 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-776a9534-8f53-42d1-aee5-7390e4f73c92 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:28:38.673464 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 529/1069] 10.222.0.57 () {56 vars in 1374 bytes} [Fri Jul 1 16:28:38 2022] GET /identity/v3/role_assignments?user.id=e0b10b53b2474b738cf28a04d0425c54&scope.project.id=08b05678e269471b80c2f94499c9d39a&role.id=84ee739146934c1ab32c501ff8093644 => generated 259 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:28:38.674299 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776a9534-8f53-42d1-aee5-7390e4f73c92 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:28:38.680508 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=100 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:28:38.762626 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 541/1070] 10.222.0.57 () {58 vars in 1282 bytes} [Fri Jul 1 16:28:38 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 01 16:28:38.769145 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bdae1d9-766e-4cc2-818d-081f4e6ae99d None admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:28:38.782121 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:28:38.814288 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776a9534-8f53-42d1-aee5-7390e4f73c92 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:28:38.866625 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56561e41-3992-482f-9f02-d17d5e4cd80e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:28:38.867444 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56561e41-3992-482f-9f02-d17d5e4cd80e None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:28:38.867709 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56561e41-3992-482f-9f02-d17d5e4cd80e None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:28:38.868148 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56561e41-3992-482f-9f02-d17d5e4cd80e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:28:38.882855 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56561e41-3992-482f-9f02-d17d5e4cd80e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:28:38.885153 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56561e41-3992-482f-9f02-d17d5e4cd80e None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:28:38.901858 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bdae1d9-766e-4cc2-818d-081f4e6ae99d None admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:28:38.906522 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application [None req-56561e41-3992-482f-9f02-d17d5e4cd80e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 16:28:38.906522 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 16:28:38.906522 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 16:28:38.906522 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 16:28:38.906522 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 16:28:38.906522 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 16:28:38.906522 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 16:28:38.906522 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 16:28:38.906522 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 16:28:38.906522 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 16:28:38.906522 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 16:28:38.906522 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 16:28:38.906522 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 01 16:28:38.906522 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 01 16:28:38.906522 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 01 16:28:38.907417 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 01 16:28:38.907417 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 16:28:38.907417 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 16:28:38.907417 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 16:28:38.907417 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 16:28:38.907417 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 16:28:38.907417 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 16:28:38.907417 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 16:28:38.907417 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application with Lock( Jul 01 16:28:38.907417 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 16:28:38.907417 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._enter() Jul 01 16:28:38.907417 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 16:28:38.907417 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 16:28:38.907417 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 16:28:38.907417 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.creator() Jul 01 16:28:38.907417 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 16:28:38.907417 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application created_value = creator( Jul 01 16:28:38.908404 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 01 16:28:38.908404 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 01 16:28:38.908404 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 16:28:38.908404 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Jul 01 16:28:38.908404 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 530/1071] 10.222.0.57 () {56 vars in 1108 bytes} [Fri Jul 1 16:28:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 16:28:38.914980 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56561e41-3992-482f-9f02-d17d5e4cd80e None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:28:38.947458 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c87d54-e1cb-44ff-8505-3effb79fb46d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:28:38.948276 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c87d54-e1cb-44ff-8505-3effb79fb46d None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:28:38.948563 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c87d54-e1cb-44ff-8505-3effb79fb46d None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:28:38.948843 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c87d54-e1cb-44ff-8505-3effb79fb46d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:28:38.963011 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c87d54-e1cb-44ff-8505-3effb79fb46d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:28:38.964481 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c87d54-e1cb-44ff-8505-3effb79fb46d None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:28:38.984309 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application [None req-94c87d54-e1cb-44ff-8505-3effb79fb46d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 16:28:38.984309 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 16:28:38.984309 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 16:28:38.984309 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 16:28:38.984309 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 16:28:38.984309 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 16:28:38.984309 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 16:28:38.984309 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 16:28:38.984309 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 16:28:38.984309 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 16:28:38.984309 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 16:28:38.984309 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 16:28:38.984309 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 01 16:28:38.984309 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 01 16:28:38.984309 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 01 16:28:38.985872 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 01 16:28:38.985872 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 16:28:38.985872 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 16:28:38.985872 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 16:28:38.985872 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 16:28:38.985872 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 16:28:38.985872 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 16:28:38.985872 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 16:28:38.985872 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application with Lock( Jul 01 16:28:38.985872 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 16:28:38.985872 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self._enter() Jul 01 16:28:38.985872 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 16:28:38.985872 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 16:28:38.985872 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 16:28:38.985872 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.creator() Jul 01 16:28:38.985872 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 16:28:38.985872 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application created_value = creator( Jul 01 16:28:38.986810 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 01 16:28:38.986810 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 01 16:28:38.986810 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 16:28:38.986810 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application Jul 01 16:28:38.986810 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 542/1072] 10.222.0.57 () {56 vars in 1108 bytes} [Fri Jul 1 16:28:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 16:28:38.992708 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c87d54-e1cb-44ff-8505-3effb79fb46d None admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:28:38.995108 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=100 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:28:39.049508 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56561e41-3992-482f-9f02-d17d5e4cd80e None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:28:39.095098 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:28:39.108631 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e5e517-e230-43a0-bc34-5bde69f0d584 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:28:39.109009 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e5e517-e230-43a0-bc34-5bde69f0d584 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:28:39.109223 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e5e517-e230-43a0-bc34-5bde69f0d584 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:28:39.109509 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e5e517-e230-43a0-bc34-5bde69f0d584 None admin] PATH_INFO: `/v3/domains` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:28:39.111104 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e5e517-e230-43a0-bc34-5bde69f0d584 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:28:39.111104 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e5e517-e230-43a0-bc34-5bde69f0d584 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:28:39.112070 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e5e517-e230-43a0-bc34-5bde69f0d584 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:28:39.133512 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 531/1073] 10.222.0.57 () {56 vars in 1109 bytes} [Fri Jul 1 16:28:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 01 16:28:39.135174 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e5e517-e230-43a0-bc34-5bde69f0d584 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:28:39.152354 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c87d54-e1cb-44ff-8505-3effb79fb46d None admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:28:39.207302 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4ce845-2aff-4367-b885-e53751c9c4a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:28:39.208519 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4ce845-2aff-4367-b885-e53751c9c4a4 None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:28:39.209665 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4ce845-2aff-4367-b885-e53751c9c4a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:28:39.210414 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4ce845-2aff-4367-b885-e53751c9c4a4 None admin] PATH_INFO: `/v3/domains` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:28:39.212229 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc4ce845-2aff-4367-b885-e53751c9c4a4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80312) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:28:39.212806 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc4ce845-2aff-4367-b885-e53751c9c4a4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:28:39.214878 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc4ce845-2aff-4367-b885-e53751c9c4a4 None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:28:39.246383 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 543/1074] 10.222.0.57 () {56 vars in 1109 bytes} [Fri Jul 1 16:28:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:28:39.282561 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e5e517-e230-43a0-bc34-5bde69f0d584 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:28:39.326716 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2244a35-6a8a-41a6-bef8-e84e971c9ba3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:28:39.327706 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2244a35-6a8a-41a6-bef8-e84e971c9ba3 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:28:39.327985 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2244a35-6a8a-41a6-bef8-e84e971c9ba3 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:28:39.328257 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2244a35-6a8a-41a6-bef8-e84e971c9ba3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:28:39.336151 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2244a35-6a8a-41a6-bef8-e84e971c9ba3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:28:39.337634 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2244a35-6a8a-41a6-bef8-e84e971c9ba3 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:28:39.353024 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application [None req-e2244a35-6a8a-41a6-bef8-e84e971c9ba3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 01 16:28:39.353024 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 16:28:39.353024 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 16:28:39.353024 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 16:28:39.353024 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 16:28:39.353024 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 16:28:39.353024 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 16:28:39.353024 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 16:28:39.353024 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 16:28:39.353024 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 16:28:39.353024 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 16:28:39.353024 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 16:28:39.353024 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 01 16:28:39.353024 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 01 16:28:39.353024 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 01 16:28:39.353988 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 01 16:28:39.353988 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 16:28:39.353988 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 16:28:39.353988 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 16:28:39.353988 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 16:28:39.353988 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 16:28:39.353988 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 16:28:39.353988 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 16:28:39.353988 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application with Lock( Jul 01 16:28:39.353988 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 16:28:39.353988 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._enter() Jul 01 16:28:39.353988 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 16:28:39.353988 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 16:28:39.353988 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 16:28:39.353988 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.creator() Jul 01 16:28:39.353988 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 16:28:39.353988 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application created_value = creator( Jul 01 16:28:39.359143 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 01 16:28:39.359143 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 01 16:28:39.359143 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 01 16:28:39.359143 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 01 16:28:39.359143 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 01 16:28:39.359143 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 01 16:28:39.359143 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 01 16:28:39.359143 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Jul 01 16:28:39.359143 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 532/1075] 10.222.0.57 () {56 vars in 1111 bytes} [Fri Jul 1 16:28:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 01 16:28:39.361911 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4ce845-2aff-4367-b885-e53751c9c4a4 None admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:28:39.437681 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=100 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:28:39.482255 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4ce845-2aff-4367-b885-e53751c9c4a4 None admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:28:39.520833 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22db6c4f-b0c5-45ad-be8b-6607a155c332 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:28:39.521790 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22db6c4f-b0c5-45ad-be8b-6607a155c332 None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:28:39.522074 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22db6c4f-b0c5-45ad-be8b-6607a155c332 None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:28:39.522366 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22db6c4f-b0c5-45ad-be8b-6607a155c332 None admin] PATH_INFO: `/v3/projects` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:28:39.523175 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22db6c4f-b0c5-45ad-be8b-6607a155c332 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80312) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:28:39.523488 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22db6c4f-b0c5-45ad-be8b-6607a155c332 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:28:39.524925 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22db6c4f-b0c5-45ad-be8b-6607a155c332 None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:28:39.540193 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 544/1076] 10.222.0.57 () {56 vars in 1148 bytes} [Fri Jul 1 16:28:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:28:39.546041 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2244a35-6a8a-41a6-bef8-e84e971c9ba3 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:28:39.615384 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:28:39.680356 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2244a35-6a8a-41a6-bef8-e84e971c9ba3 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:28:39.718732 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95e2a78-6d00-46f9-9b4c-bacff04787ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:28:39.719627 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95e2a78-6d00-46f9-9b4c-bacff04787ab None admin] REQUEST_METHOD: `PUT` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:28:39.719908 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95e2a78-6d00-46f9-9b4c-bacff04787ab None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:28:39.720181 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95e2a78-6d00-46f9-9b4c-bacff04787ab None admin] PATH_INFO: `/v3/projects/e0788e6725a84e57a4174284c1845755/users/7a50f745b31941b7a7265c0fc852f48e/roles/c3a62bbbb749474aad6655e693c8963d` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:28:39.722510 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b95e2a78-6d00-46f9-9b4c-bacff04787ab None admin] RBAC: Authorizing `identity:create_grant(project_id=e0788e6725a84e57a4174284c1845755, user_id=7a50f745b31941b7a7265c0fc852f48e, role_id=c3a62bbbb749474aad6655e693c8963d)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:28:39.723928 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b95e2a78-6d00-46f9-9b4c-bacff04787ab None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:28:39.738538 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 533/1077] 10.222.0.57 () {58 vars in 1439 bytes} [Fri Jul 1 16:28:39 2022] PUT /identity/v3/projects/e0788e6725a84e57a4174284c1845755/users/7a50f745b31941b7a7265c0fc852f48e/roles/c3a62bbbb749474aad6655e693c8963d => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 01 16:28:40.782627 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1a5998-f6f5-451f-bcec-5ebb598f46f7 None None] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:28:40.783265 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1a5998-f6f5-451f-bcec-5ebb598f46f7 None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:28:40.783265 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1a5998-f6f5-451f-bcec-5ebb598f46f7 None None] PATH_INFO: `/` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:28:40.785823 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 545/1078] 10.222.0.57 () {54 vars in 926 bytes} [Fri Jul 1 16:28:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 16:28:40.791079 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a1349c-8206-4a6a-8d1a-784716f4e398 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:28:40.791490 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a1349c-8206-4a6a-8d1a-784716f4e398 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:28:40.791634 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a1349c-8206-4a6a-8d1a-784716f4e398 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:28:40.809267 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-32a1349c-8206-4a6a-8d1a-784716f4e398 None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:28:40.810625 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-32a1349c-8206-4a6a-8d1a-784716f4e398 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:28:40.890978 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 534/1079] 10.222.0.57 () {58 vars in 1023 bytes} [Fri Jul 1 16:28:40 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 01 16:28:40.914867 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1a5998-f6f5-451f-bcec-5ebb598f46f7 None None] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:28:41.035556 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1a5998-f6f5-451f-bcec-5ebb598f46f7 None None] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:28:41.059012 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=100 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:28:41.071433 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf81189c-143f-490f-bd8d-b876df2a9049 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:28:41.072229 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf81189c-143f-490f-bd8d-b876df2a9049 None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:28:41.072512 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf81189c-143f-490f-bd8d-b876df2a9049 None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:28:41.072786 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf81189c-143f-490f-bd8d-b876df2a9049 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:28:41.084349 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf81189c-143f-490f-bd8d-b876df2a9049 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 01 16:28:41.093586 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application [None req-cf81189c-143f-490f-bd8d-b876df2a9049 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 01 16:28:41.093586 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 16:28:41.093586 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 16:28:41.093586 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 16:28:41.093586 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 16:28:41.093586 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 16:28:41.093586 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 16:28:41.093586 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 16:28:41.093586 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 16:28:41.093586 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 16:28:41.093586 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 16:28:41.093586 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 16:28:41.093586 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 01 16:28:41.093586 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 01 16:28:41.093586 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 01 16:28:41.093586 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 01 16:28:41.094818 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 01 16:28:41.094818 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 01 16:28:41.094818 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 01 16:28:41.094818 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 01 16:28:41.094818 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 16:28:41.094818 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 16:28:41.094818 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 16:28:41.094818 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 16:28:41.094818 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 16:28:41.094818 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 16:28:41.094818 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 16:28:41.094818 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application with Lock( Jul 01 16:28:41.094818 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 16:28:41.094818 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self._enter() Jul 01 16:28:41.094818 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 16:28:41.094818 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 16:28:41.095757 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 16:28:41.095757 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.creator() Jul 01 16:28:41.095757 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 16:28:41.095757 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application created_value = creator( Jul 01 16:28:41.095757 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 01 16:28:41.095757 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 01 16:28:41.095757 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 01 16:28:41.095757 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 01 16:28:41.095757 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 01 16:28:41.095757 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 01 16:28:41.095757 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 01 16:28:41.095757 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application Jul 01 16:28:41.095757 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 546/1080] 10.222.0.57 () {56 vars in 1096 bytes} [Fri Jul 1 16:28:40 2022] GET /identity/v3/roles/admin => generated 83 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 16:28:41.101810 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a1349c-8206-4a6a-8d1a-784716f4e398 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:28:41.143869 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:28:41.203654 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a1349c-8206-4a6a-8d1a-784716f4e398 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:28:41.239291 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76272388-f094-4826-8084-e6cd37cb1f5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:28:41.239950 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76272388-f094-4826-8084-e6cd37cb1f5c None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:28:41.240232 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76272388-f094-4826-8084-e6cd37cb1f5c None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:28:41.240510 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76272388-f094-4826-8084-e6cd37cb1f5c None admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:28:41.241262 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76272388-f094-4826-8084-e6cd37cb1f5c None admin] RBAC: Adding query filter params (name=admin) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:28:41.241573 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76272388-f094-4826-8084-e6cd37cb1f5c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:28:41.242984 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76272388-f094-4826-8084-e6cd37cb1f5c None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:28:41.251254 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 535/1081] 10.222.0.57 () {56 vars in 1099 bytes} [Fri Jul 1 16:28:41 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 01 16:28:41.262134 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf81189c-143f-490f-bd8d-b876df2a9049 None admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:28:41.368275 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf81189c-143f-490f-bd8d-b876df2a9049 None admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:28:41.404879 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70be5363-fedd-478d-bac6-c8b364e350e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:28:41.405760 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70be5363-fedd-478d-bac6-c8b364e350e4 None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:28:41.406043 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70be5363-fedd-478d-bac6-c8b364e350e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:28:41.406320 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70be5363-fedd-478d-bac6-c8b364e350e4 None admin] PATH_INFO: `/v3/users/e0b10b53b2474b738cf28a04d0425c54` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:28:41.407549 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70be5363-fedd-478d-bac6-c8b364e350e4 None admin] RBAC: Authorizing `identity:get_user(user_id=e0b10b53b2474b738cf28a04d0425c54)` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:28:41.410274 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70be5363-fedd-478d-bac6-c8b364e350e4 None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:28:41.413521 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 547/1082] 10.222.0.57 () {56 vars in 1177 bytes} [Fri Jul 1 16:28:41 2022] GET /identity/v3/users/e0b10b53b2474b738cf28a04d0425c54 => generated 285 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:28:41.418730 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76272388-f094-4826-8084-e6cd37cb1f5c None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:28:41.530736 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76272388-f094-4826-8084-e6cd37cb1f5c None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:28:41.532349 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:28:41.571945 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5013f0d3-1f59-4936-aa8d-3ea91979b9f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:28:41.572814 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5013f0d3-1f59-4936-aa8d-3ea91979b9f2 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:28:41.573097 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5013f0d3-1f59-4936-aa8d-3ea91979b9f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:28:41.573380 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5013f0d3-1f59-4936-aa8d-3ea91979b9f2 None admin] PATH_INFO: `/v3/projects/08b05678e269471b80c2f94499c9d39a` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:28:41.576484 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5013f0d3-1f59-4936-aa8d-3ea91979b9f2 None admin] RBAC: Authorizing `identity:get_project(project_id=08b05678e269471b80c2f94499c9d39a)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:28:41.578232 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5013f0d3-1f59-4936-aa8d-3ea91979b9f2 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:28:41.580034 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 536/1083] 10.222.0.57 () {56 vars in 1186 bytes} [Fri Jul 1 16:28:41 2022] GET /identity/v3/projects/08b05678e269471b80c2f94499c9d39a => generated 311 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:28:41.586686 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70be5363-fedd-478d-bac6-c8b364e350e4 None admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:28:41.638988 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=100 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:28:41.689524 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70be5363-fedd-478d-bac6-c8b364e350e4 None admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:28:41.724177 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3916daf7-4bea-4867-9d74-ddc066aa06f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:28:41.725058 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3916daf7-4bea-4867-9d74-ddc066aa06f0 None admin] REQUEST_METHOD: `PUT` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:28:41.725340 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3916daf7-4bea-4867-9d74-ddc066aa06f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:28:41.725617 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3916daf7-4bea-4867-9d74-ddc066aa06f0 None admin] PATH_INFO: `/v3/projects/08b05678e269471b80c2f94499c9d39a/users/e0b10b53b2474b738cf28a04d0425c54/roles/84ee739146934c1ab32c501ff8093644` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:28:41.727564 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3916daf7-4bea-4867-9d74-ddc066aa06f0 None admin] RBAC: Authorizing `identity:create_grant(project_id=08b05678e269471b80c2f94499c9d39a, user_id=e0b10b53b2474b738cf28a04d0425c54, role_id=84ee739146934c1ab32c501ff8093644)` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:28:41.728980 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3916daf7-4bea-4867-9d74-ddc066aa06f0 None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:28:41.741928 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 548/1084] 10.222.0.57 () {58 vars in 1439 bytes} [Fri Jul 1 16:28:41 2022] PUT /identity/v3/projects/08b05678e269471b80c2f94499c9d39a/users/e0b10b53b2474b738cf28a04d0425c54/roles/84ee739146934c1ab32c501ff8093644 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 16:28:41.767875 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca2957d-ace2-42ad-b302-9be508d948f2 None None] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:28:41.768822 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca2957d-ace2-42ad-b302-9be508d948f2 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:28:41.769577 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca2957d-ace2-42ad-b302-9be508d948f2 None None] PATH_INFO: `/` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:28:41.770747 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 537/1085] 10.222.0.57 () {54 vars in 926 bytes} [Fri Jul 1 16:28:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 16:28:41.776749 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d57ef0f-f1c8-4ecd-9a78-089b7dfa189c None None] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:28:41.777036 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d57ef0f-f1c8-4ecd-9a78-089b7dfa189c None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:28:41.777316 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d57ef0f-f1c8-4ecd-9a78-089b7dfa189c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:28:41.794974 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.auth.core [None req-8d57ef0f-f1c8-4ecd-9a78-089b7dfa189c None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80312) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:28:41.796215 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-8d57ef0f-f1c8-4ecd-9a78-089b7dfa189c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:28:41.869580 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 549/1086] 10.222.0.57 () {58 vars in 1023 bytes} [Fri Jul 1 16:28:41 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 01 16:28:41.894723 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca2957d-ace2-42ad-b302-9be508d948f2 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:28:41.998907 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca2957d-ace2-42ad-b302-9be508d948f2 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:28:42.033694 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e50aa2-b4a0-4f95-b4f3-0b2112b99bc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:28:42.034597 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e50aa2-b4a0-4f95-b4f3-0b2112b99bc5 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:28:42.034878 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e50aa2-b4a0-4f95-b4f3-0b2112b99bc5 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:28:42.035156 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e50aa2-b4a0-4f95-b4f3-0b2112b99bc5 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:28:42.045674 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e50aa2-b4a0-4f95-b4f3-0b2112b99bc5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 01 16:28:42.054391 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application [None req-b5e50aa2-b4a0-4f95-b4f3-0b2112b99bc5 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 01 16:28:42.054391 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 16:28:42.054391 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 16:28:42.054391 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 16:28:42.054391 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 16:28:42.054391 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 16:28:42.054391 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 16:28:42.054391 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 16:28:42.054391 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 16:28:42.054391 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 16:28:42.054391 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 16:28:42.054391 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 16:28:42.054391 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 01 16:28:42.054391 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 01 16:28:42.054391 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 01 16:28:42.054391 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 01 16:28:42.055486 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 01 16:28:42.055486 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 01 16:28:42.055486 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 01 16:28:42.055486 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 01 16:28:42.055486 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 16:28:42.055486 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 16:28:42.055486 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 16:28:42.055486 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 16:28:42.055486 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 16:28:42.055486 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 16:28:42.055486 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 16:28:42.055486 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application with Lock( Jul 01 16:28:42.055486 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 16:28:42.055486 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._enter() Jul 01 16:28:42.055486 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 16:28:42.055486 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 16:28:42.056722 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 16:28:42.056722 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.creator() Jul 01 16:28:42.056722 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 16:28:42.056722 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application created_value = creator( Jul 01 16:28:42.056722 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 01 16:28:42.056722 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 01 16:28:42.056722 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 01 16:28:42.056722 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 01 16:28:42.056722 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 01 16:28:42.056722 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 01 16:28:42.056722 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 01 16:28:42.056722 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Jul 01 16:28:42.056722 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 538/1087] 10.222.0.57 () {56 vars in 1120 bytes} [Fri Jul 1 16:28:41 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 16:28:42.061534 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d57ef0f-f1c8-4ecd-9a78-089b7dfa189c None None] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:28:42.103103 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:28:42.160955 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d57ef0f-f1c8-4ecd-9a78-089b7dfa189c None None] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:28:42.194809 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404714ca-02c9-49b6-86e4-15bfb2ea3426 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:28:42.195479 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404714ca-02c9-49b6-86e4-15bfb2ea3426 None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:28:42.195765 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404714ca-02c9-49b6-86e4-15bfb2ea3426 None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:28:42.196042 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404714ca-02c9-49b6-86e4-15bfb2ea3426 None admin] PATH_INFO: `/v3/roles` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:28:42.196808 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-404714ca-02c9-49b6-86e4-15bfb2ea3426 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80312) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:28:42.197114 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-404714ca-02c9-49b6-86e4-15bfb2ea3426 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:28:42.198510 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-404714ca-02c9-49b6-86e4-15bfb2ea3426 None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:28:42.207084 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 550/1088] 10.222.0.57 () {56 vars in 1115 bytes} [Fri Jul 1 16:28:42 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:28:42.212685 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e50aa2-b4a0-4f95-b4f3-0b2112b99bc5 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:28:42.243776 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:28:42.314128 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e50aa2-b4a0-4f95-b4f3-0b2112b99bc5 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:28:42.348076 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e8b90e-976e-49c6-b04c-af3560170787 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:28:42.348854 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e8b90e-976e-49c6-b04c-af3560170787 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:28:42.349134 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e8b90e-976e-49c6-b04c-af3560170787 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:28:42.349407 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e8b90e-976e-49c6-b04c-af3560170787 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:28:42.359740 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e8b90e-976e-49c6-b04c-af3560170787 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:28:42.361169 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e8b90e-976e-49c6-b04c-af3560170787 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:28:42.378656 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application [None req-a2e8b90e-976e-49c6-b04c-af3560170787 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 16:28:42.378656 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 16:28:42.378656 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 16:28:42.378656 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 16:28:42.378656 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 16:28:42.378656 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 16:28:42.378656 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 16:28:42.378656 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 16:28:42.378656 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 16:28:42.378656 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 16:28:42.378656 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 16:28:42.378656 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 16:28:42.378656 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 01 16:28:42.378656 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 01 16:28:42.378656 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 01 16:28:42.379896 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 01 16:28:42.379896 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 16:28:42.379896 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 16:28:42.379896 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 16:28:42.379896 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 16:28:42.379896 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 16:28:42.379896 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 16:28:42.379896 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 16:28:42.379896 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application with Lock( Jul 01 16:28:42.379896 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 16:28:42.379896 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._enter() Jul 01 16:28:42.379896 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 16:28:42.379896 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 16:28:42.379896 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 16:28:42.379896 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.creator() Jul 01 16:28:42.379896 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 16:28:42.379896 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application created_value = creator( Jul 01 16:28:42.380845 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 01 16:28:42.380845 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 01 16:28:42.380845 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 16:28:42.380845 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Jul 01 16:28:42.380845 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 539/1089] 10.222.0.57 () {56 vars in 1108 bytes} [Fri Jul 1 16:28:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 16:28:42.385778 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404714ca-02c9-49b6-86e4-15bfb2ea3426 None admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:28:42.484942 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404714ca-02c9-49b6-86e4-15bfb2ea3426 None admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:28:42.486385 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=100 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:28:42.519971 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47349eee-a1d3-4951-8300-7cc589115c00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:28:42.520625 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47349eee-a1d3-4951-8300-7cc589115c00 None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:28:42.520906 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47349eee-a1d3-4951-8300-7cc589115c00 None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:28:42.521181 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47349eee-a1d3-4951-8300-7cc589115c00 None admin] PATH_INFO: `/v3/domains` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:28:42.521896 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47349eee-a1d3-4951-8300-7cc589115c00 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80312) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:28:42.522210 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47349eee-a1d3-4951-8300-7cc589115c00 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:28:42.523592 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47349eee-a1d3-4951-8300-7cc589115c00 None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:28:42.535922 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 551/1090] 10.222.0.57 () {56 vars in 1109 bytes} [Fri Jul 1 16:28:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 01 16:28:42.541692 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e8b90e-976e-49c6-b04c-af3560170787 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:28:42.622440 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:28:42.640661 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e8b90e-976e-49c6-b04c-af3560170787 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:28:42.674069 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45e4997-e92e-4404-9c55-165dbce01fb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:28:42.674976 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45e4997-e92e-4404-9c55-165dbce01fb6 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:28:42.675264 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45e4997-e92e-4404-9c55-165dbce01fb6 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:28:42.675533 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45e4997-e92e-4404-9c55-165dbce01fb6 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:28:42.683056 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a45e4997-e92e-4404-9c55-165dbce01fb6 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:28:42.684495 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a45e4997-e92e-4404-9c55-165dbce01fb6 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:28:42.698563 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application [None req-a45e4997-e92e-4404-9c55-165dbce01fb6 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 01 16:28:42.698563 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 16:28:42.698563 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 16:28:42.698563 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 16:28:42.698563 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 16:28:42.698563 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 16:28:42.698563 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 16:28:42.698563 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 16:28:42.698563 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 16:28:42.698563 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 16:28:42.698563 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 16:28:42.698563 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 16:28:42.698563 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 01 16:28:42.698563 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 01 16:28:42.698563 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 01 16:28:42.698563 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 01 16:28:42.699686 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 16:28:42.699686 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 16:28:42.699686 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 01 16:28:42.699686 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 01 16:28:42.699686 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 01 16:28:42.699686 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 01 16:28:42.699686 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 16:28:42.699686 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 16:28:42.699686 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 16:28:42.699686 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 16:28:42.699686 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 16:28:42.699686 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application with Lock( Jul 01 16:28:42.699686 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 16:28:42.699686 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._enter() Jul 01 16:28:42.699686 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 16:28:42.699686 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 16:28:42.699686 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 16:28:42.699686 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.creator() Jul 01 16:28:42.700640 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 16:28:42.700640 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application created_value = creator( Jul 01 16:28:42.700640 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 01 16:28:42.700640 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 01 16:28:42.700640 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 01 16:28:42.700640 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 01 16:28:42.700640 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 01 16:28:42.700640 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 01 16:28:42.700640 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 01 16:28:42.700640 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Jul 01 16:28:42.700640 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 540/1091] 10.222.0.57 () {56 vars in 1093 bytes} [Fri Jul 1 16:28:42 2022] GET /identity/v3/users/nova => generated 82 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 16:28:42.706527 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47349eee-a1d3-4951-8300-7cc589115c00 None admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:28:42.806051 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47349eee-a1d3-4951-8300-7cc589115c00 None admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:28:42.843749 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd187dcd-e91d-43ce-a27f-34b38b9789b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:28:42.844427 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd187dcd-e91d-43ce-a27f-34b38b9789b8 None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:28:42.844709 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd187dcd-e91d-43ce-a27f-34b38b9789b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:28:42.844984 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd187dcd-e91d-43ce-a27f-34b38b9789b8 None admin] PATH_INFO: `/v3/users` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:28:42.845823 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd187dcd-e91d-43ce-a27f-34b38b9789b8 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80312) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:28:42.846132 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd187dcd-e91d-43ce-a27f-34b38b9789b8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:28:42.848296 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd187dcd-e91d-43ce-a27f-34b38b9789b8 None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:28:42.867638 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 552/1092] 10.222.0.57 () {56 vars in 1133 bytes} [Fri Jul 1 16:28:42 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:28:42.873259 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45e4997-e92e-4404-9c55-165dbce01fb6 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:28:42.973116 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45e4997-e92e-4404-9c55-165dbce01fb6 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:28:43.006785 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92e6487-549a-4335-9678-2f7bb11b07fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:28:43.007427 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92e6487-549a-4335-9678-2f7bb11b07fe None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:28:43.007708 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92e6487-549a-4335-9678-2f7bb11b07fe None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:28:43.007979 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92e6487-549a-4335-9678-2f7bb11b07fe None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:28:43.019624 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:28:43.075318 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a92e6487-549a-4335-9678-2f7bb11b07fe None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:28:43.076631 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a92e6487-549a-4335-9678-2f7bb11b07fe None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:28:43.172423 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 541/1093] 10.222.0.57 () {58 vars in 1282 bytes} [Fri Jul 1 16:28:42 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 01 16:28:43.178772 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd187dcd-e91d-43ce-a27f-34b38b9789b8 None admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:28:43.199955 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=100 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:28:43.277811 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd187dcd-e91d-43ce-a27f-34b38b9789b8 None admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:28:43.311622 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23f369c-3a57-4fd0-ab06-1e73e996314d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:28:43.312415 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23f369c-3a57-4fd0-ab06-1e73e996314d None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:28:43.312702 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23f369c-3a57-4fd0-ab06-1e73e996314d None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:28:43.312979 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23f369c-3a57-4fd0-ab06-1e73e996314d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:28:43.323211 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c23f369c-3a57-4fd0-ab06-1e73e996314d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:28:43.324670 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c23f369c-3a57-4fd0-ab06-1e73e996314d None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:28:43.342501 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application [None req-c23f369c-3a57-4fd0-ab06-1e73e996314d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 16:28:43.342501 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 16:28:43.342501 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 16:28:43.342501 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 16:28:43.342501 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 16:28:43.342501 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 16:28:43.342501 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 16:28:43.342501 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 16:28:43.342501 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 16:28:43.342501 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 16:28:43.342501 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 16:28:43.342501 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 16:28:43.342501 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 01 16:28:43.342501 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 01 16:28:43.342501 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 01 16:28:43.343592 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 01 16:28:43.343592 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 16:28:43.343592 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 16:28:43.343592 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 16:28:43.343592 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 16:28:43.343592 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 16:28:43.343592 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 16:28:43.343592 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 16:28:43.343592 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application with Lock( Jul 01 16:28:43.343592 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 16:28:43.343592 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self._enter() Jul 01 16:28:43.343592 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 16:28:43.343592 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 16:28:43.343592 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 16:28:43.343592 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.creator() Jul 01 16:28:43.343592 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 16:28:43.343592 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application created_value = creator( Jul 01 16:28:43.344532 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 01 16:28:43.344532 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 01 16:28:43.344532 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 16:28:43.344532 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application Jul 01 16:28:43.344532 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 553/1094] 10.222.0.57 () {56 vars in 1108 bytes} [Fri Jul 1 16:28:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 16:28:43.349971 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92e6487-549a-4335-9678-2f7bb11b07fe None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:28:43.393775 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:28:43.450758 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92e6487-549a-4335-9678-2f7bb11b07fe None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:28:43.484741 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000c2731-76f1-483a-a8d1-14ec11b7b965 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:28:43.485400 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000c2731-76f1-483a-a8d1-14ec11b7b965 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:28:43.485686 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000c2731-76f1-483a-a8d1-14ec11b7b965 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:28:43.485964 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000c2731-76f1-483a-a8d1-14ec11b7b965 None admin] PATH_INFO: `/v3/domains` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:28:43.486729 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000c2731-76f1-483a-a8d1-14ec11b7b965 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:28:43.487139 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000c2731-76f1-483a-a8d1-14ec11b7b965 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:28:43.488535 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000c2731-76f1-483a-a8d1-14ec11b7b965 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:28:43.499780 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 542/1095] 10.222.0.57 () {56 vars in 1109 bytes} [Fri Jul 1 16:28:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:28:43.505591 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23f369c-3a57-4fd0-ab06-1e73e996314d None admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:28:43.575050 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=100 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:28:43.606453 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23f369c-3a57-4fd0-ab06-1e73e996314d None admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:28:43.651040 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb53c7d-57ae-400e-89bd-fcad2c0f66ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:28:43.652222 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb53c7d-57ae-400e-89bd-fcad2c0f66ff None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:28:43.652626 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb53c7d-57ae-400e-89bd-fcad2c0f66ff None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:28:43.652904 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb53c7d-57ae-400e-89bd-fcad2c0f66ff None admin] PATH_INFO: `/v3/projects/service` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:28:43.661783 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb53c7d-57ae-400e-89bd-fcad2c0f66ff None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:28:43.664116 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb53c7d-57ae-400e-89bd-fcad2c0f66ff None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:28:43.681614 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application [None req-2eb53c7d-57ae-400e-89bd-fcad2c0f66ff None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 01 16:28:43.681614 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 16:28:43.681614 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 16:28:43.681614 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 16:28:43.681614 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 16:28:43.681614 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 16:28:43.681614 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 16:28:43.681614 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 16:28:43.681614 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 16:28:43.681614 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 16:28:43.681614 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 16:28:43.681614 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 16:28:43.681614 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 01 16:28:43.681614 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 01 16:28:43.681614 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 01 16:28:43.682774 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 01 16:28:43.682774 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 16:28:43.682774 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 16:28:43.682774 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 16:28:43.682774 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 16:28:43.682774 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 16:28:43.682774 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 16:28:43.682774 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 16:28:43.682774 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application with Lock( Jul 01 16:28:43.682774 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 16:28:43.682774 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self._enter() Jul 01 16:28:43.682774 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 16:28:43.682774 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 16:28:43.682774 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 16:28:43.682774 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.creator() Jul 01 16:28:43.682774 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 16:28:43.682774 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application created_value = creator( Jul 01 16:28:43.683914 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 01 16:28:43.683914 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 01 16:28:43.683914 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 01 16:28:43.683914 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 01 16:28:43.683914 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 01 16:28:43.683914 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 01 16:28:43.683914 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 01 16:28:43.683914 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application Jul 01 16:28:43.686504 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 554/1096] 10.222.0.57 () {56 vars in 1111 bytes} [Fri Jul 1 16:28:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 16:28:43.690476 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000c2731-76f1-483a-a8d1-14ec11b7b965 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:28:43.794902 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000c2731-76f1-483a-a8d1-14ec11b7b965 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:28:43.796448 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:28:43.833415 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e789c1-de31-479e-8665-d3aead3eb8c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:28:43.834394 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e789c1-de31-479e-8665-d3aead3eb8c0 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:28:43.834838 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e789c1-de31-479e-8665-d3aead3eb8c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:28:43.835267 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e789c1-de31-479e-8665-d3aead3eb8c0 None admin] PATH_INFO: `/v3/projects` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:28:43.836215 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e789c1-de31-479e-8665-d3aead3eb8c0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:28:43.836332 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec7ca72-52a2-49fa-9cc9-3b3f6ef8bf99 None None] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:28:43.836756 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec7ca72-52a2-49fa-9cc9-3b3f6ef8bf99 None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:28:43.836853 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec7ca72-52a2-49fa-9cc9-3b3f6ef8bf99 None None] PATH_INFO: `/` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:28:43.837293 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e789c1-de31-479e-8665-d3aead3eb8c0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:28:43.838196 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 555/1097] 10.222.0.57 () {54 vars in 926 bytes} [Fri Jul 1 16:28:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 16:28:43.838906 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e789c1-de31-479e-8665-d3aead3eb8c0 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:28:43.843535 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ad13ee-e707-4cb4-945e-529488c2f6f2 None None] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:28:43.843821 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ad13ee-e707-4cb4-945e-529488c2f6f2 None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:28:43.844102 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ad13ee-e707-4cb4-945e-529488c2f6f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:28:43.851886 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 543/1098] 10.222.0.57 () {56 vars in 1148 bytes} [Fri Jul 1 16:28:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:28:43.857715 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e789c1-de31-479e-8665-d3aead3eb8c0 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:28:43.861766 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.auth.core [None req-f8ad13ee-e707-4cb4-945e-529488c2f6f2 None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80312) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:28:43.863208 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-f8ad13ee-e707-4cb4-945e-529488c2f6f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:28:43.930306 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 556/1099] 10.222.0.57 () {58 vars in 1023 bytes} [Fri Jul 1 16:28:43 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 01 16:28:43.955285 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ad13ee-e707-4cb4-945e-529488c2f6f2 None None] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:28:43.958249 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e789c1-de31-479e-8665-d3aead3eb8c0 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:28:43.993250 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32fe8cc-7982-441b-8dbf-199b945f4ba8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:28:43.993947 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32fe8cc-7982-441b-8dbf-199b945f4ba8 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:28:43.994193 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32fe8cc-7982-441b-8dbf-199b945f4ba8 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:28:43.994507 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32fe8cc-7982-441b-8dbf-199b945f4ba8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:28:43.995357 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b32fe8cc-7982-441b-8dbf-199b945f4ba8 None admin] RBAC: Adding query filter params (role.id=c3a62bbbb749474aad6655e693c8963d, scope.project.id=e0788e6725a84e57a4174284c1845755, user.id=7a50f745b31941b7a7265c0fc852f48e) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:28:43.995698 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b32fe8cc-7982-441b-8dbf-199b945f4ba8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:28:43.997146 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b32fe8cc-7982-441b-8dbf-199b945f4ba8 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:28:44.003808 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 544/1100] 10.222.0.57 () {56 vars in 1374 bytes} [Fri Jul 1 16:28:43 2022] GET /identity/v3/role_assignments?user.id=7a50f745b31941b7a7265c0fc852f48e&scope.project.id=e0788e6725a84e57a4174284c1845755&role.id=c3a62bbbb749474aad6655e693c8963d => generated 608 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 01 16:28:44.022570 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=100 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:28:44.066518 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ad13ee-e707-4cb4-945e-529488c2f6f2 None None] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:28:44.107273 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e968a0bf-c500-486c-bf22-10eff114954b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:28:44.108240 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e968a0bf-c500-486c-bf22-10eff114954b None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:28:44.108660 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e968a0bf-c500-486c-bf22-10eff114954b None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:28:44.109120 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e968a0bf-c500-486c-bf22-10eff114954b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:28:44.126568 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e968a0bf-c500-486c-bf22-10eff114954b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 01 16:28:44.135153 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application [None req-e968a0bf-c500-486c-bf22-10eff114954b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 01 16:28:44.135153 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 16:28:44.135153 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 16:28:44.135153 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 16:28:44.135153 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 16:28:44.135153 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 16:28:44.135153 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 16:28:44.135153 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 16:28:44.135153 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 16:28:44.135153 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 16:28:44.135153 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 16:28:44.135153 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 16:28:44.135153 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 01 16:28:44.135153 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 01 16:28:44.135153 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 01 16:28:44.135153 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 01 16:28:44.136155 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 01 16:28:44.136155 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 01 16:28:44.136155 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 01 16:28:44.136155 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 01 16:28:44.136155 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 16:28:44.136155 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 16:28:44.136155 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 16:28:44.136155 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 16:28:44.136155 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 16:28:44.136155 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 16:28:44.136155 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 16:28:44.136155 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application with Lock( Jul 01 16:28:44.136155 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 16:28:44.136155 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self._enter() Jul 01 16:28:44.136155 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 16:28:44.136155 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 16:28:44.137008 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 16:28:44.137008 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.creator() Jul 01 16:28:44.137008 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 16:28:44.137008 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application created_value = creator( Jul 01 16:28:44.137008 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 01 16:28:44.137008 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 01 16:28:44.137008 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 01 16:28:44.137008 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 01 16:28:44.137008 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 01 16:28:44.137008 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 01 16:28:44.137008 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 01 16:28:44.137008 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application Jul 01 16:28:44.137008 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 557/1101] 10.222.0.57 () {56 vars in 1096 bytes} [Fri Jul 1 16:28:43 2022] GET /identity/v3/roles/admin => generated 83 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 01 16:28:44.142590 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32fe8cc-7982-441b-8dbf-199b945f4ba8 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:28:44.171499 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:28:44.248816 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32fe8cc-7982-441b-8dbf-199b945f4ba8 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:28:44.285113 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000cd0fc-49f7-4f31-9142-26f7d5feb8a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:28:44.285796 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000cd0fc-49f7-4f31-9142-26f7d5feb8a3 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:28:44.286084 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000cd0fc-49f7-4f31-9142-26f7d5feb8a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:28:44.286433 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000cd0fc-49f7-4f31-9142-26f7d5feb8a3 None admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:28:44.287198 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000cd0fc-49f7-4f31-9142-26f7d5feb8a3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:28:44.287513 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000cd0fc-49f7-4f31-9142-26f7d5feb8a3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:28:44.288904 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000cd0fc-49f7-4f31-9142-26f7d5feb8a3 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:28:44.298008 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 545/1102] 10.222.0.57 () {56 vars in 1099 bytes} [Fri Jul 1 16:28:44 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:28:44.304044 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e968a0bf-c500-486c-bf22-10eff114954b None admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:28:44.408598 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e968a0bf-c500-486c-bf22-10eff114954b None admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:28:44.432757 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=100 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:28:44.446661 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d190f8-c4c5-4ca7-9190-af93a996f0a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:28:44.447540 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d190f8-c4c5-4ca7-9190-af93a996f0a6 None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:28:44.447820 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d190f8-c4c5-4ca7-9190-af93a996f0a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:28:44.448091 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d190f8-c4c5-4ca7-9190-af93a996f0a6 None admin] PATH_INFO: `/v3/users/e0b10b53b2474b738cf28a04d0425c54` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:28:44.449260 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d190f8-c4c5-4ca7-9190-af93a996f0a6 None admin] RBAC: Authorizing `identity:get_user(user_id=e0b10b53b2474b738cf28a04d0425c54)` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:28:44.450724 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d190f8-c4c5-4ca7-9190-af93a996f0a6 None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:28:44.452634 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 558/1103] 10.222.0.57 () {56 vars in 1177 bytes} [Fri Jul 1 16:28:44 2022] GET /identity/v3/users/e0b10b53b2474b738cf28a04d0425c54 => generated 285 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:28:44.458668 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000cd0fc-49f7-4f31-9142-26f7d5feb8a3 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:28:44.551089 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:28:44.564393 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000cd0fc-49f7-4f31-9142-26f7d5feb8a3 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:28:44.602028 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70841e0b-e8df-4626-9da8-92720d3bd76e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:28:44.602749 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70841e0b-e8df-4626-9da8-92720d3bd76e None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:28:44.603034 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70841e0b-e8df-4626-9da8-92720d3bd76e None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:28:44.603314 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70841e0b-e8df-4626-9da8-92720d3bd76e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:28:44.677433 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70841e0b-e8df-4626-9da8-92720d3bd76e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:28:44.678863 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70841e0b-e8df-4626-9da8-92720d3bd76e None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:28:44.766795 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:28:44.780856 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 546/1104] 10.222.0.57 () {58 vars in 1282 bytes} [Fri Jul 1 16:28:44 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 01 16:28:44.787005 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d190f8-c4c5-4ca7-9190-af93a996f0a6 None admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:28:44.892599 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d190f8-c4c5-4ca7-9190-af93a996f0a6 None admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:28:44.925839 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd585d2a-6096-4e6c-967e-2ccd919854ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:28:44.926727 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd585d2a-6096-4e6c-967e-2ccd919854ce None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:28:44.927009 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd585d2a-6096-4e6c-967e-2ccd919854ce None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:28:44.927282 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd585d2a-6096-4e6c-967e-2ccd919854ce None admin] PATH_INFO: `/v3/projects/08b05678e269471b80c2f94499c9d39a` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:28:44.928530 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd585d2a-6096-4e6c-967e-2ccd919854ce None admin] RBAC: Authorizing `identity:get_project(project_id=08b05678e269471b80c2f94499c9d39a)` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:28:44.929909 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd585d2a-6096-4e6c-967e-2ccd919854ce None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:28:44.931647 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 559/1105] 10.222.0.57 () {56 vars in 1186 bytes} [Fri Jul 1 16:28:44 2022] GET /identity/v3/projects/08b05678e269471b80c2f94499c9d39a => generated 311 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:28:44.937800 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70841e0b-e8df-4626-9da8-92720d3bd76e None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:28:45.035042 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70841e0b-e8df-4626-9da8-92720d3bd76e None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:28:45.067532 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868a9f68-2c14-43b0-b81a-7e99cba3dd75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:28:45.068175 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868a9f68-2c14-43b0-b81a-7e99cba3dd75 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:28:45.068463 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868a9f68-2c14-43b0-b81a-7e99cba3dd75 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:28:45.068744 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868a9f68-2c14-43b0-b81a-7e99cba3dd75 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:28:45.069571 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868a9f68-2c14-43b0-b81a-7e99cba3dd75 None admin] RBAC: Adding query filter params (role.id=84ee739146934c1ab32c501ff8093644, scope.project.id=08b05678e269471b80c2f94499c9d39a, user.id=e0b10b53b2474b738cf28a04d0425c54) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:28:45.069888 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868a9f68-2c14-43b0-b81a-7e99cba3dd75 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:28:45.071333 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868a9f68-2c14-43b0-b81a-7e99cba3dd75 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:28:45.077263 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 547/1106] 10.222.0.57 () {56 vars in 1374 bytes} [Fri Jul 1 16:28:44 2022] GET /identity/v3/role_assignments?user.id=e0b10b53b2474b738cf28a04d0425c54&scope.project.id=08b05678e269471b80c2f94499c9d39a&role.id=84ee739146934c1ab32c501ff8093644 => generated 608 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:28:47.305696 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9c7221-2ad5-4d8e-a781-001af3711df0 None None] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:28:47.306271 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9c7221-2ad5-4d8e-a781-001af3711df0 None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:28:47.306271 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9c7221-2ad5-4d8e-a781-001af3711df0 None None] PATH_INFO: `/` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:28:47.307791 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 560/1107] 10.222.0.57 () {54 vars in 926 bytes} [Fri Jul 1 16:28:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 16:28:47.313145 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85756ab0-02ac-4096-bd97-7c09fd0e8c55 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:28:47.313597 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85756ab0-02ac-4096-bd97-7c09fd0e8c55 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:28:47.313716 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85756ab0-02ac-4096-bd97-7c09fd0e8c55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:28:47.331173 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-85756ab0-02ac-4096-bd97-7c09fd0e8c55 None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:28:47.332632 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-85756ab0-02ac-4096-bd97-7c09fd0e8c55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:28:47.381803 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:28:47.402468 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 548/1108] 10.222.0.57 () {58 vars in 1023 bytes} [Fri Jul 1 16:28:47 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 01 16:28:47.428003 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9c7221-2ad5-4d8e-a781-001af3711df0 None None] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:28:47.487241 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=100 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:28:47.532028 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9c7221-2ad5-4d8e-a781-001af3711df0 None None] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:28:47.567926 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d620e297-cd0e-4ad7-9c71-d6056f8bd185 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:28:47.568638 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d620e297-cd0e-4ad7-9c71-d6056f8bd185 None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:28:47.568925 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d620e297-cd0e-4ad7-9c71-d6056f8bd185 None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:28:47.569204 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d620e297-cd0e-4ad7-9c71-d6056f8bd185 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:28:47.570508 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d620e297-cd0e-4ad7-9c71-d6056f8bd185 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:28:47.571835 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d620e297-cd0e-4ad7-9c71-d6056f8bd185 None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:28:47.573470 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 561/1109] 10.222.0.57 () {56 vars in 1108 bytes} [Fri Jul 1 16:28:47 2022] GET /identity/v3/domains/default => generated 199 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:28:47.579663 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85756ab0-02ac-4096-bd97-7c09fd0e8c55 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:28:47.681920 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85756ab0-02ac-4096-bd97-7c09fd0e8c55 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:28:47.716524 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90051708-15a4-4766-ab9f-b8cdd49f9924 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:28:47.717192 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90051708-15a4-4766-ab9f-b8cdd49f9924 None admin] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:28:47.717481 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90051708-15a4-4766-ab9f-b8cdd49f9924 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:28:47.717758 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90051708-15a4-4766-ab9f-b8cdd49f9924 None admin] PATH_INFO: `/v3/users` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:28:47.718508 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90051708-15a4-4766-ab9f-b8cdd49f9924 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:28:47.719934 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90051708-15a4-4766-ab9f-b8cdd49f9924 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:28:47.739152 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 549/1110] 10.222.0.57 () {60 vars in 1132 bytes} [Fri Jul 1 16:28:47 2022] POST /identity/v3/users => generated 286 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 16:28:49.862968 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5887727c-9916-4498-b324-9e97824b661f None None] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:28:49.863441 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5887727c-9916-4498-b324-9e97824b661f None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:28:49.863616 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5887727c-9916-4498-b324-9e97824b661f None None] PATH_INFO: `/` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:28:49.864895 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 562/1111] 10.222.0.57 () {54 vars in 926 bytes} [Fri Jul 1 16:28:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 16:28:49.869971 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-accdd18c-6b46-4777-833a-c16032675395 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:28:49.870355 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-accdd18c-6b46-4777-833a-c16032675395 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:28:49.870570 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-accdd18c-6b46-4777-833a-c16032675395 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:28:49.887500 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-accdd18c-6b46-4777-833a-c16032675395 None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:28:49.888681 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-accdd18c-6b46-4777-833a-c16032675395 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:28:49.926534 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:28:49.965657 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 550/1112] 10.222.0.57 () {58 vars in 1023 bytes} [Fri Jul 1 16:28:49 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 01 16:28:49.990160 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5887727c-9916-4498-b324-9e97824b661f None None] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:28:50.090151 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5887727c-9916-4498-b324-9e97824b661f None None] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:28:50.116115 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=100 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:28:50.125042 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f9f6b4-ea6a-48be-a3ad-a3b8099a5901 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:28:50.125907 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f9f6b4-ea6a-48be-a3ad-a3b8099a5901 None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:28:50.126194 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f9f6b4-ea6a-48be-a3ad-a3b8099a5901 None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:28:50.126503 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f9f6b4-ea6a-48be-a3ad-a3b8099a5901 None admin] PATH_INFO: `/v3/roles/a8e5c73da81d4f0eb7b92c7058e500fe` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:28:50.128023 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f9f6b4-ea6a-48be-a3ad-a3b8099a5901 None admin] RBAC: Authorizing `identity:get_role(role_id=a8e5c73da81d4f0eb7b92c7058e500fe)` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:28:50.129389 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f9f6b4-ea6a-48be-a3ad-a3b8099a5901 None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:28:50.130797 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 563/1113] 10.222.0.57 () {56 vars in 1177 bytes} [Fri Jul 1 16:28:49 2022] GET /identity/v3/roles/a8e5c73da81d4f0eb7b92c7058e500fe => generated 223 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:28:50.136669 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-accdd18c-6b46-4777-833a-c16032675395 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:28:50.244291 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-accdd18c-6b46-4777-833a-c16032675395 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:28:50.278546 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed122d1-fd70-4ad6-aff9-8c976135c523 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:28:50.279406 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed122d1-fd70-4ad6-aff9-8c976135c523 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:28:50.279687 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed122d1-fd70-4ad6-aff9-8c976135c523 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:28:50.279958 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed122d1-fd70-4ad6-aff9-8c976135c523 None admin] PATH_INFO: `/v3/users/d60caa6a2ca44abea32ee25630a4f5c6` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:28:50.293212 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed122d1-fd70-4ad6-aff9-8c976135c523 None admin] RBAC: Authorizing `identity:get_user(user_id=d60caa6a2ca44abea32ee25630a4f5c6)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:28:50.294681 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed122d1-fd70-4ad6-aff9-8c976135c523 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:28:50.296325 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 551/1114] 10.222.0.57 () {56 vars in 1177 bytes} [Fri Jul 1 16:28:50 2022] GET /identity/v3/users/d60caa6a2ca44abea32ee25630a4f5c6 => generated 286 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:28:50.302834 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f9f6b4-ea6a-48be-a3ad-a3b8099a5901 None admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:28:50.402792 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f9f6b4-ea6a-48be-a3ad-a3b8099a5901 None admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:28:50.435483 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ef97e2-be23-4dcc-aeff-7658927913d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:28:50.436119 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ef97e2-be23-4dcc-aeff-7658927913d4 None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:28:50.436406 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ef97e2-be23-4dcc-aeff-7658927913d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:28:50.436683 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ef97e2-be23-4dcc-aeff-7658927913d4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:28:50.495389 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=100 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:28:50.504083 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ef97e2-be23-4dcc-aeff-7658927913d4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:28:50.505535 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ef97e2-be23-4dcc-aeff-7658927913d4 None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:28:50.605575 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 564/1115] 10.222.0.57 () {58 vars in 1282 bytes} [Fri Jul 1 16:28:50 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 01 16:28:50.612076 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed122d1-fd70-4ad6-aff9-8c976135c523 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:28:50.638514 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:28:50.709607 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed122d1-fd70-4ad6-aff9-8c976135c523 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:28:50.744236 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f4c239-a800-4d1f-8b04-fe237eaf2991 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:28:50.745089 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f4c239-a800-4d1f-8b04-fe237eaf2991 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:28:50.745370 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f4c239-a800-4d1f-8b04-fe237eaf2991 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:28:50.745645 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f4c239-a800-4d1f-8b04-fe237eaf2991 None admin] PATH_INFO: `/v3/projects/08b05678e269471b80c2f94499c9d39a` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:28:50.746833 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f4c239-a800-4d1f-8b04-fe237eaf2991 None admin] RBAC: Authorizing `identity:get_project(project_id=08b05678e269471b80c2f94499c9d39a)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:28:50.748245 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f4c239-a800-4d1f-8b04-fe237eaf2991 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:28:50.749925 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 552/1116] 10.222.0.57 () {56 vars in 1186 bytes} [Fri Jul 1 16:28:50 2022] GET /identity/v3/projects/08b05678e269471b80c2f94499c9d39a => generated 311 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:28:50.756649 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ef97e2-be23-4dcc-aeff-7658927913d4 None admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:28:50.850333 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=100 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:28:50.857301 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ef97e2-be23-4dcc-aeff-7658927913d4 None admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:28:50.892257 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1519162d-9b40-49ee-8a47-a483fe3c3845 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:28:50.893452 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1519162d-9b40-49ee-8a47-a483fe3c3845 None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:28:50.893728 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1519162d-9b40-49ee-8a47-a483fe3c3845 None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:28:50.894008 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1519162d-9b40-49ee-8a47-a483fe3c3845 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:28:50.894871 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1519162d-9b40-49ee-8a47-a483fe3c3845 None admin] RBAC: Adding query filter params (role.id=a8e5c73da81d4f0eb7b92c7058e500fe, scope.project.id=08b05678e269471b80c2f94499c9d39a, user.id=d60caa6a2ca44abea32ee25630a4f5c6) {{(pid=80312) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:28:50.895193 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1519162d-9b40-49ee-8a47-a483fe3c3845 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:28:50.896620 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1519162d-9b40-49ee-8a47-a483fe3c3845 None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:28:50.901943 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 565/1117] 10.222.0.57 () {56 vars in 1374 bytes} [Fri Jul 1 16:28:50 2022] GET /identity/v3/role_assignments?user.id=d60caa6a2ca44abea32ee25630a4f5c6&scope.project.id=08b05678e269471b80c2f94499c9d39a&role.id=a8e5c73da81d4f0eb7b92c7058e500fe => generated 259 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:28:52.915107 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c0a248-fb33-4c5d-bac9-764bc45e664f None None] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:28:52.915622 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c0a248-fb33-4c5d-bac9-764bc45e664f None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:28:52.915744 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c0a248-fb33-4c5d-bac9-764bc45e664f None None] PATH_INFO: `/` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:28:52.917053 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 553/1118] 10.222.0.57 () {54 vars in 926 bytes} [Fri Jul 1 16:28:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 16:28:52.922328 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178b04cd-12a1-4560-88d0-931f67d0dca0 None None] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:28:52.922763 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178b04cd-12a1-4560-88d0-931f67d0dca0 None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:28:52.922921 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178b04cd-12a1-4560-88d0-931f67d0dca0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:28:52.941061 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.auth.core [None req-178b04cd-12a1-4560-88d0-931f67d0dca0 None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80312) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:28:52.942021 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-178b04cd-12a1-4560-88d0-931f67d0dca0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:28:53.012155 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 566/1119] 10.222.0.57 () {58 vars in 1023 bytes} [Fri Jul 1 16:28:52 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 01 16:28:53.035801 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c0a248-fb33-4c5d-bac9-764bc45e664f None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:28:53.127535 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:28:53.135281 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c0a248-fb33-4c5d-bac9-764bc45e664f None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:28:53.169044 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd290d04-1fb6-4294-a6c7-9b7ff5850bf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:28:53.169888 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd290d04-1fb6-4294-a6c7-9b7ff5850bf3 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:28:53.170173 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd290d04-1fb6-4294-a6c7-9b7ff5850bf3 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:28:53.170479 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd290d04-1fb6-4294-a6c7-9b7ff5850bf3 None admin] PATH_INFO: `/v3/roles/a8e5c73da81d4f0eb7b92c7058e500fe` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:28:53.171928 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd290d04-1fb6-4294-a6c7-9b7ff5850bf3 None admin] RBAC: Authorizing `identity:get_role(role_id=a8e5c73da81d4f0eb7b92c7058e500fe)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:28:53.173291 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd290d04-1fb6-4294-a6c7-9b7ff5850bf3 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:28:53.175185 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 554/1120] 10.222.0.57 () {56 vars in 1177 bytes} [Fri Jul 1 16:28:53 2022] GET /identity/v3/roles/a8e5c73da81d4f0eb7b92c7058e500fe => generated 223 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:28:53.181089 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178b04cd-12a1-4560-88d0-931f67d0dca0 None None] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:28:53.275750 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=100 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:28:53.288254 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178b04cd-12a1-4560-88d0-931f67d0dca0 None None] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:28:53.321717 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9902a2-7a79-47c8-a938-ecf1dd2ec08a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:28:53.322602 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9902a2-7a79-47c8-a938-ecf1dd2ec08a None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:28:53.322887 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9902a2-7a79-47c8-a938-ecf1dd2ec08a None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:28:53.323165 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9902a2-7a79-47c8-a938-ecf1dd2ec08a None admin] PATH_INFO: `/v3/users/d60caa6a2ca44abea32ee25630a4f5c6` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:28:53.324332 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9902a2-7a79-47c8-a938-ecf1dd2ec08a None admin] RBAC: Authorizing `identity:get_user(user_id=d60caa6a2ca44abea32ee25630a4f5c6)` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:28:53.325771 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9902a2-7a79-47c8-a938-ecf1dd2ec08a None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:28:53.327688 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 567/1121] 10.222.0.57 () {56 vars in 1177 bytes} [Fri Jul 1 16:28:53 2022] GET /identity/v3/users/d60caa6a2ca44abea32ee25630a4f5c6 => generated 286 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:28:53.333794 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd290d04-1fb6-4294-a6c7-9b7ff5850bf3 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:28:53.431626 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd290d04-1fb6-4294-a6c7-9b7ff5850bf3 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:28:53.466228 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f51214-7fe9-423f-ab08-52db13003642 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:28:53.467091 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f51214-7fe9-423f-ab08-52db13003642 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:28:53.467375 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f51214-7fe9-423f-ab08-52db13003642 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:28:53.467653 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f51214-7fe9-423f-ab08-52db13003642 None admin] PATH_INFO: `/v3/projects/08b05678e269471b80c2f94499c9d39a` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:28:53.468823 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f51214-7fe9-423f-ab08-52db13003642 None admin] RBAC: Authorizing `identity:get_project(project_id=08b05678e269471b80c2f94499c9d39a)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:28:53.470244 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f51214-7fe9-423f-ab08-52db13003642 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:28:53.472472 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 555/1122] 10.222.0.57 () {56 vars in 1186 bytes} [Fri Jul 1 16:28:53 2022] GET /identity/v3/projects/08b05678e269471b80c2f94499c9d39a => generated 311 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:28:53.481063 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9902a2-7a79-47c8-a938-ecf1dd2ec08a None admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:28:53.584403 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9902a2-7a79-47c8-a938-ecf1dd2ec08a None admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:28:53.617992 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eefdc7d-6c1b-475a-a071-22d2e5d635cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:28:53.618905 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eefdc7d-6c1b-475a-a071-22d2e5d635cf None admin] REQUEST_METHOD: `PUT` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:28:53.619189 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eefdc7d-6c1b-475a-a071-22d2e5d635cf None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:28:53.619463 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eefdc7d-6c1b-475a-a071-22d2e5d635cf None admin] PATH_INFO: `/v3/projects/08b05678e269471b80c2f94499c9d39a/users/d60caa6a2ca44abea32ee25630a4f5c6/roles/a8e5c73da81d4f0eb7b92c7058e500fe` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:28:53.621626 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eefdc7d-6c1b-475a-a071-22d2e5d635cf None admin] RBAC: Authorizing `identity:create_grant(project_id=08b05678e269471b80c2f94499c9d39a, user_id=d60caa6a2ca44abea32ee25630a4f5c6, role_id=a8e5c73da81d4f0eb7b92c7058e500fe)` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:28:53.623082 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eefdc7d-6c1b-475a-a071-22d2e5d635cf None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:28:53.635955 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 568/1123] 10.222.0.57 () {58 vars in 1439 bytes} [Fri Jul 1 16:28:53 2022] PUT /identity/v3/projects/08b05678e269471b80c2f94499c9d39a/users/d60caa6a2ca44abea32ee25630a4f5c6/roles/a8e5c73da81d4f0eb7b92c7058e500fe => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 16:28:55.663275 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc553595-e57d-473d-b394-15cfdcd6b28a None None] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:28:55.663835 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc553595-e57d-473d-b394-15cfdcd6b28a None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:28:55.663835 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc553595-e57d-473d-b394-15cfdcd6b28a None None] PATH_INFO: `/` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:28:55.665213 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 556/1124] 10.222.0.57 () {54 vars in 926 bytes} [Fri Jul 1 16:28:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 16:28:55.670311 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73aefc4-3254-4942-a457-3fd26a462b50 None None] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:28:55.670718 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73aefc4-3254-4942-a457-3fd26a462b50 None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:28:55.671028 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73aefc4-3254-4942-a457-3fd26a462b50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:28:55.688529 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.auth.core [None req-b73aefc4-3254-4942-a457-3fd26a462b50 None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80312) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:28:55.690279 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-b73aefc4-3254-4942-a457-3fd26a462b50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:28:55.700511 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:28:55.755476 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 569/1125] 10.222.0.57 () {58 vars in 1023 bytes} [Fri Jul 1 16:28:55 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 01 16:28:55.779226 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc553595-e57d-473d-b394-15cfdcd6b28a None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:28:55.809088 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:28:55.881781 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc553595-e57d-473d-b394-15cfdcd6b28a None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:28:55.918254 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f5bee7-570e-45e3-ae12-88ee2af01cb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:28:55.921167 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f5bee7-570e-45e3-ae12-88ee2af01cb9 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:28:55.921470 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f5bee7-570e-45e3-ae12-88ee2af01cb9 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:28:55.921758 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f5bee7-570e-45e3-ae12-88ee2af01cb9 None admin] PATH_INFO: `/v3/roles/a8e5c73da81d4f0eb7b92c7058e500fe` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:28:55.924468 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f5bee7-570e-45e3-ae12-88ee2af01cb9 None admin] RBAC: Authorizing `identity:get_role(role_id=a8e5c73da81d4f0eb7b92c7058e500fe)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:28:55.926180 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f5bee7-570e-45e3-ae12-88ee2af01cb9 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:28:55.927638 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 557/1126] 10.222.0.57 () {56 vars in 1177 bytes} [Fri Jul 1 16:28:55 2022] GET /identity/v3/roles/a8e5c73da81d4f0eb7b92c7058e500fe => generated 223 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:28:55.933426 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73aefc4-3254-4942-a457-3fd26a462b50 None None] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:28:56.030229 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73aefc4-3254-4942-a457-3fd26a462b50 None None] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:28:56.063208 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1016aa3f-ff67-4497-af89-463f1bee4a90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:28:56.064583 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1016aa3f-ff67-4497-af89-463f1bee4a90 None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:28:56.064874 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1016aa3f-ff67-4497-af89-463f1bee4a90 None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:28:56.065151 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1016aa3f-ff67-4497-af89-463f1bee4a90 None admin] PATH_INFO: `/v3/users/d60caa6a2ca44abea32ee25630a4f5c6` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:28:56.066332 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1016aa3f-ff67-4497-af89-463f1bee4a90 None admin] RBAC: Authorizing `identity:get_user(user_id=d60caa6a2ca44abea32ee25630a4f5c6)` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:28:56.067785 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1016aa3f-ff67-4497-af89-463f1bee4a90 None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:28:56.069406 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 570/1127] 10.222.0.57 () {56 vars in 1177 bytes} [Fri Jul 1 16:28:55 2022] GET /identity/v3/users/d60caa6a2ca44abea32ee25630a4f5c6 => generated 286 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 01 16:28:56.075104 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f5bee7-570e-45e3-ae12-88ee2af01cb9 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:28:56.170271 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:28:56.174886 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f5bee7-570e-45e3-ae12-88ee2af01cb9 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:28:56.209091 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12bea403-e748-467d-abfa-b0b3e663977a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:28:56.209724 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bea403-e748-467d-abfa-b0b3e663977a None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:28:56.210006 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bea403-e748-467d-abfa-b0b3e663977a None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:28:56.210280 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bea403-e748-467d-abfa-b0b3e663977a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:28:56.279573 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12bea403-e748-467d-abfa-b0b3e663977a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:28:56.281025 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12bea403-e748-467d-abfa-b0b3e663977a None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:28:56.374243 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:28:56.379346 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 558/1128] 10.222.0.57 () {58 vars in 1282 bytes} [Fri Jul 1 16:28:56 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 01 16:28:56.384958 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1016aa3f-ff67-4497-af89-463f1bee4a90 None admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:28:56.400095 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=100 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:28:56.481853 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1016aa3f-ff67-4497-af89-463f1bee4a90 None admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:28:56.514870 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119f855b-4ee6-41e1-9b3b-3da1a5bc2c1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:28:56.515733 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119f855b-4ee6-41e1-9b3b-3da1a5bc2c1f None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:28:56.516018 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119f855b-4ee6-41e1-9b3b-3da1a5bc2c1f None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:28:56.516322 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119f855b-4ee6-41e1-9b3b-3da1a5bc2c1f None admin] PATH_INFO: `/v3/projects/08b05678e269471b80c2f94499c9d39a` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:28:56.517678 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-119f855b-4ee6-41e1-9b3b-3da1a5bc2c1f None admin] RBAC: Authorizing `identity:get_project(project_id=08b05678e269471b80c2f94499c9d39a)` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:28:56.519177 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-119f855b-4ee6-41e1-9b3b-3da1a5bc2c1f None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:28:56.520926 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 571/1129] 10.222.0.57 () {56 vars in 1186 bytes} [Fri Jul 1 16:28:56 2022] GET /identity/v3/projects/08b05678e269471b80c2f94499c9d39a => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:28:56.527070 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12bea403-e748-467d-abfa-b0b3e663977a None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:28:56.625804 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12bea403-e748-467d-abfa-b0b3e663977a None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:28:56.659598 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444d3d42-87ed-446b-88e0-15ef8be7e8ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:28:56.660245 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444d3d42-87ed-446b-88e0-15ef8be7e8ba None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:28:56.660533 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444d3d42-87ed-446b-88e0-15ef8be7e8ba None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:28:56.660813 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444d3d42-87ed-446b-88e0-15ef8be7e8ba None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:28:56.661640 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-444d3d42-87ed-446b-88e0-15ef8be7e8ba None admin] RBAC: Adding query filter params (role.id=a8e5c73da81d4f0eb7b92c7058e500fe, scope.project.id=08b05678e269471b80c2f94499c9d39a, user.id=d60caa6a2ca44abea32ee25630a4f5c6) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:28:56.661959 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-444d3d42-87ed-446b-88e0-15ef8be7e8ba None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:28:56.663444 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-444d3d42-87ed-446b-88e0-15ef8be7e8ba None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:28:56.669136 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 559/1130] 10.222.0.57 () {56 vars in 1374 bytes} [Fri Jul 1 16:28:56 2022] GET /identity/v3/role_assignments?user.id=d60caa6a2ca44abea32ee25630a4f5c6&scope.project.id=08b05678e269471b80c2f94499c9d39a&role.id=a8e5c73da81d4f0eb7b92c7058e500fe => generated 608 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:28:58.750011 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4247407-a421-466d-8e11-71abc77e8680 None None] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:28:58.750619 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4247407-a421-466d-8e11-71abc77e8680 None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:28:58.750619 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4247407-a421-466d-8e11-71abc77e8680 None None] PATH_INFO: `/` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:28:58.752700 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 572/1131] 10.222.0.57 () {54 vars in 926 bytes} [Fri Jul 1 16:28:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 16:28:58.757733 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca2eb2c-e68a-431a-a8de-af09ea61dd7a None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:28:58.758120 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca2eb2c-e68a-431a-a8de-af09ea61dd7a None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:28:58.758279 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca2eb2c-e68a-431a-a8de-af09ea61dd7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:28:58.775057 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-1ca2eb2c-e68a-431a-a8de-af09ea61dd7a None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:28:58.776215 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-1ca2eb2c-e68a-431a-a8de-af09ea61dd7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:28:58.830923 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:28:58.841564 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 560/1132] 10.222.0.57 () {58 vars in 1023 bytes} [Fri Jul 1 16:28:58 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 01 16:28:58.867051 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4247407-a421-466d-8e11-71abc77e8680 None None] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:28:58.946986 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=100 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:28:58.967016 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4247407-a421-466d-8e11-71abc77e8680 None None] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:28:59.000413 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66b50a6-3bcf-4908-9600-f7a2c4bb7a11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:28:59.001132 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66b50a6-3bcf-4908-9600-f7a2c4bb7a11 None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:28:59.001468 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66b50a6-3bcf-4908-9600-f7a2c4bb7a11 None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:28:59.001867 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66b50a6-3bcf-4908-9600-f7a2c4bb7a11 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:28:59.003157 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b66b50a6-3bcf-4908-9600-f7a2c4bb7a11 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:28:59.004580 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b66b50a6-3bcf-4908-9600-f7a2c4bb7a11 None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:28:59.006187 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 573/1133] 10.222.0.57 () {56 vars in 1108 bytes} [Fri Jul 1 16:28:58 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:28:59.012020 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca2eb2c-e68a-431a-a8de-af09ea61dd7a None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:28:59.109378 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca2eb2c-e68a-431a-a8de-af09ea61dd7a None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:28:59.143671 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3807463-2ba8-410f-9ed5-696445321c03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:28:59.144330 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3807463-2ba8-410f-9ed5-696445321c03 None admin] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:28:59.144618 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3807463-2ba8-410f-9ed5-696445321c03 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:28:59.144899 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3807463-2ba8-410f-9ed5-696445321c03 None admin] PATH_INFO: `/v3/projects` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:28:59.145615 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3807463-2ba8-410f-9ed5-696445321c03 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:28:59.147162 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3807463-2ba8-410f-9ed5-696445321c03 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:28:59.171550 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 561/1134] 10.222.0.57 () {60 vars in 1141 bytes} [Fri Jul 1 16:28:59 2022] POST /identity/v3/projects => generated 311 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 16:29:01.323483 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792fc085-99dc-4acf-a4dc-f6956a9947af None None] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:29:01.324011 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792fc085-99dc-4acf-a4dc-f6956a9947af None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:29:01.324124 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792fc085-99dc-4acf-a4dc-f6956a9947af None None] PATH_INFO: `/` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:29:01.325434 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 574/1135] 10.222.0.57 () {54 vars in 926 bytes} [Fri Jul 1 16:29:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 16:29:01.331261 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80732df4-1ff1-480c-95da-596d8fb0c946 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:29:01.331687 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80732df4-1ff1-480c-95da-596d8fb0c946 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:29:01.331777 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80732df4-1ff1-480c-95da-596d8fb0c946 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:29:01.348730 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-80732df4-1ff1-480c-95da-596d8fb0c946 None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:29:01.349926 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-80732df4-1ff1-480c-95da-596d8fb0c946 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:29:01.389752 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:29:01.418411 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 562/1136] 10.222.0.57 () {58 vars in 1023 bytes} [Fri Jul 1 16:29:01 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 01 16:29:01.444399 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792fc085-99dc-4acf-a4dc-f6956a9947af None None] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:29:01.546169 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792fc085-99dc-4acf-a4dc-f6956a9947af None None] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:29:01.580493 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e314e2-ef61-44b0-9ba4-e92b82990eb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:29:01.581205 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e314e2-ef61-44b0-9ba4-e92b82990eb6 None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:29:01.581484 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e314e2-ef61-44b0-9ba4-e92b82990eb6 None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:29:01.581758 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e314e2-ef61-44b0-9ba4-e92b82990eb6 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:29:01.582949 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e314e2-ef61-44b0-9ba4-e92b82990eb6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:29:01.584373 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e314e2-ef61-44b0-9ba4-e92b82990eb6 None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:29:01.585982 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 575/1137] 10.222.0.57 () {56 vars in 1108 bytes} [Fri Jul 1 16:29:01 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:29:01.591860 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80732df4-1ff1-480c-95da-596d8fb0c946 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:29:01.687162 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80732df4-1ff1-480c-95da-596d8fb0c946 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:29:01.720635 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1dd4fdc-ac0c-4979-a901-7973ee83890e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:29:01.721303 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1dd4fdc-ac0c-4979-a901-7973ee83890e None admin] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:29:01.721569 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1dd4fdc-ac0c-4979-a901-7973ee83890e None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:29:01.721843 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1dd4fdc-ac0c-4979-a901-7973ee83890e None admin] PATH_INFO: `/v3/users` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:29:01.722581 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1dd4fdc-ac0c-4979-a901-7973ee83890e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:29:01.724004 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1dd4fdc-ac0c-4979-a901-7973ee83890e None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:29:01.743788 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 563/1138] 10.222.0.57 () {60 vars in 1132 bytes} [Fri Jul 1 16:29:01 2022] POST /identity/v3/users => generated 286 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 16:29:03.809700 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f11f6e3-af3d-4135-b1b4-03f78fde0621 None None] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:29:03.810274 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f11f6e3-af3d-4135-b1b4-03f78fde0621 None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:29:03.810274 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f11f6e3-af3d-4135-b1b4-03f78fde0621 None None] PATH_INFO: `/` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:29:03.811636 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 576/1139] 10.222.0.57 () {54 vars in 926 bytes} [Fri Jul 1 16:29:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 16:29:03.817731 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898078c6-3358-4e88-a349-3088b53e0d58 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:29:03.818160 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898078c6-3358-4e88-a349-3088b53e0d58 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:29:03.818292 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898078c6-3358-4e88-a349-3088b53e0d58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:29:03.835352 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-898078c6-3358-4e88-a349-3088b53e0d58 None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:29:03.836557 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-898078c6-3358-4e88-a349-3088b53e0d58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:29:03.859958 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:29:03.904182 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 564/1140] 10.222.0.57 () {58 vars in 1023 bytes} [Fri Jul 1 16:29:03 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 01 16:29:03.928999 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f11f6e3-af3d-4135-b1b4-03f78fde0621 None None] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:29:03.948377 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=100 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:29:04.038583 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f11f6e3-af3d-4135-b1b4-03f78fde0621 None None] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:29:04.072440 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b982d38a-1374-4b3a-9656-1d891d676563 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:29:04.073156 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b982d38a-1374-4b3a-9656-1d891d676563 None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:29:04.073443 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b982d38a-1374-4b3a-9656-1d891d676563 None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:29:04.073719 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b982d38a-1374-4b3a-9656-1d891d676563 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:29:04.084215 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b982d38a-1374-4b3a-9656-1d891d676563 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 01 16:29:04.091956 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application [None req-b982d38a-1374-4b3a-9656-1d891d676563 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 01 16:29:04.091956 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 16:29:04.091956 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 16:29:04.091956 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 16:29:04.091956 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 16:29:04.091956 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 16:29:04.091956 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 16:29:04.091956 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 16:29:04.091956 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 16:29:04.091956 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 16:29:04.091956 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 16:29:04.091956 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 16:29:04.091956 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 01 16:29:04.091956 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 01 16:29:04.091956 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 01 16:29:04.091956 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 01 16:29:04.093064 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 01 16:29:04.093064 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 01 16:29:04.093064 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 01 16:29:04.093064 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 01 16:29:04.093064 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 16:29:04.093064 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 16:29:04.093064 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 16:29:04.093064 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 16:29:04.093064 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 16:29:04.093064 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 16:29:04.093064 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 16:29:04.093064 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application with Lock( Jul 01 16:29:04.093064 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 16:29:04.093064 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self._enter() Jul 01 16:29:04.093064 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 16:29:04.093064 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 16:29:04.094028 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 16:29:04.094028 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.creator() Jul 01 16:29:04.094028 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 16:29:04.094028 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application created_value = creator( Jul 01 16:29:04.094028 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 01 16:29:04.094028 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 01 16:29:04.094028 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 01 16:29:04.094028 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 01 16:29:04.094028 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 01 16:29:04.094028 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 01 16:29:04.094028 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 01 16:29:04.094028 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application Jul 01 16:29:04.094028 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 577/1141] 10.222.0.57 () {56 vars in 1096 bytes} [Fri Jul 1 16:29:03 2022] GET /identity/v3/roles/admin => generated 83 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 16:29:04.099999 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898078c6-3358-4e88-a349-3088b53e0d58 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:29:04.197684 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898078c6-3358-4e88-a349-3088b53e0d58 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:29:04.232439 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75272185-dce5-455f-a14b-0075bbbf9beb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:29:04.233107 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75272185-dce5-455f-a14b-0075bbbf9beb None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:29:04.233395 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75272185-dce5-455f-a14b-0075bbbf9beb None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:29:04.233677 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75272185-dce5-455f-a14b-0075bbbf9beb None admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:29:04.234460 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75272185-dce5-455f-a14b-0075bbbf9beb None admin] RBAC: Adding query filter params (name=admin) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:29:04.234775 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75272185-dce5-455f-a14b-0075bbbf9beb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:29:04.236812 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75272185-dce5-455f-a14b-0075bbbf9beb None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:29:04.245005 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 565/1142] 10.222.0.57 () {56 vars in 1099 bytes} [Fri Jul 1 16:29:04 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:29:04.250754 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b982d38a-1374-4b3a-9656-1d891d676563 None admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:29:04.332080 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=100 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:29:04.354897 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b982d38a-1374-4b3a-9656-1d891d676563 None admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:29:04.388066 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca946877-396c-44dc-a2c1-52f6926b4c12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:29:04.388929 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca946877-396c-44dc-a2c1-52f6926b4c12 None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:29:04.389214 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca946877-396c-44dc-a2c1-52f6926b4c12 None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:29:04.389493 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca946877-396c-44dc-a2c1-52f6926b4c12 None admin] PATH_INFO: `/v3/users/f3e86fdc531542f1968af23b4ecac21b` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:29:04.402737 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca946877-396c-44dc-a2c1-52f6926b4c12 None admin] RBAC: Authorizing `identity:get_user(user_id=f3e86fdc531542f1968af23b4ecac21b)` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:29:04.404178 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca946877-396c-44dc-a2c1-52f6926b4c12 None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:29:04.405837 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 578/1143] 10.222.0.57 () {56 vars in 1177 bytes} [Fri Jul 1 16:29:04 2022] GET /identity/v3/users/f3e86fdc531542f1968af23b4ecac21b => generated 286 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:29:04.412191 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75272185-dce5-455f-a14b-0075bbbf9beb None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:29:04.436144 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:29:04.509673 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75272185-dce5-455f-a14b-0075bbbf9beb None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:29:04.543192 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52591d32-ccec-4f2f-b692-eabe2c71833d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:29:04.543834 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52591d32-ccec-4f2f-b692-eabe2c71833d None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:29:04.544120 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52591d32-ccec-4f2f-b692-eabe2c71833d None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:29:04.544401 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52591d32-ccec-4f2f-b692-eabe2c71833d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:29:04.609675 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52591d32-ccec-4f2f-b692-eabe2c71833d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:29:04.611116 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52591d32-ccec-4f2f-b692-eabe2c71833d None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:29:04.632983 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:29:04.707020 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 566/1144] 10.222.0.57 () {58 vars in 1282 bytes} [Fri Jul 1 16:29:04 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 01 16:29:04.713025 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca946877-396c-44dc-a2c1-52f6926b4c12 None admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:29:04.812003 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca946877-396c-44dc-a2c1-52f6926b4c12 None admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:29:04.847879 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b03da0-fb3b-40e2-b7fb-913207f8f64f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:29:04.848645 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b03da0-fb3b-40e2-b7fb-913207f8f64f None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:29:04.848929 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b03da0-fb3b-40e2-b7fb-913207f8f64f None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:29:04.849207 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b03da0-fb3b-40e2-b7fb-913207f8f64f None admin] PATH_INFO: `/v3/projects/2754a147cd5a4b40944818e7ff9bb837` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:29:04.850401 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b03da0-fb3b-40e2-b7fb-913207f8f64f None admin] RBAC: Authorizing `identity:get_project(project_id=2754a147cd5a4b40944818e7ff9bb837)` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:29:04.851829 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b03da0-fb3b-40e2-b7fb-913207f8f64f None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:29:04.853531 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 579/1145] 10.222.0.57 () {56 vars in 1186 bytes} [Fri Jul 1 16:29:04 2022] GET /identity/v3/projects/2754a147cd5a4b40944818e7ff9bb837 => generated 311 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:29:04.859671 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52591d32-ccec-4f2f-b692-eabe2c71833d None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:29:04.957618 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52591d32-ccec-4f2f-b692-eabe2c71833d None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:29:04.979543 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:29:04.991528 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248d4b71-1ed5-406d-a86d-c1a847f4a77b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:29:04.992174 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248d4b71-1ed5-406d-a86d-c1a847f4a77b None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:29:04.992463 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248d4b71-1ed5-406d-a86d-c1a847f4a77b None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:29:04.992739 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248d4b71-1ed5-406d-a86d-c1a847f4a77b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:29:04.993563 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-248d4b71-1ed5-406d-a86d-c1a847f4a77b None admin] RBAC: Adding query filter params (role.id=84ee739146934c1ab32c501ff8093644, scope.project.id=2754a147cd5a4b40944818e7ff9bb837, user.id=f3e86fdc531542f1968af23b4ecac21b) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:29:04.993876 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-248d4b71-1ed5-406d-a86d-c1a847f4a77b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:29:04.995350 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-248d4b71-1ed5-406d-a86d-c1a847f4a77b None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:29:05.000680 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 567/1146] 10.222.0.57 () {56 vars in 1374 bytes} [Fri Jul 1 16:29:04 2022] GET /identity/v3/role_assignments?user.id=f3e86fdc531542f1968af23b4ecac21b&scope.project.id=2754a147cd5a4b40944818e7ff9bb837&role.id=84ee739146934c1ab32c501ff8093644 => generated 259 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:29:07.000518 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c649ab4-9280-4288-a9e5-322555199eb7 None None] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:29:07.001096 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c649ab4-9280-4288-a9e5-322555199eb7 None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:29:07.001096 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c649ab4-9280-4288-a9e5-322555199eb7 None None] PATH_INFO: `/` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:29:07.003300 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 580/1147] 10.222.0.57 () {54 vars in 926 bytes} [Fri Jul 1 16:29:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 16:29:07.008501 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bad167-cfde-4ae9-b9d6-14a6b77e8e32 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:29:07.008880 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bad167-cfde-4ae9-b9d6-14a6b77e8e32 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:29:07.009063 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bad167-cfde-4ae9-b9d6-14a6b77e8e32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:29:07.026481 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-f6bad167-cfde-4ae9-b9d6-14a6b77e8e32 None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:29:07.027668 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-f6bad167-cfde-4ae9-b9d6-14a6b77e8e32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:29:07.093881 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 568/1148] 10.222.0.57 () {58 vars in 1023 bytes} [Fri Jul 1 16:29:07 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 01 16:29:07.117290 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c649ab4-9280-4288-a9e5-322555199eb7 None None] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:29:07.121068 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=100 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:29:07.215865 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c649ab4-9280-4288-a9e5-322555199eb7 None None] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:29:07.248727 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f874338-aa5e-4a84-b666-8eefe8d2b424 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:29:07.249505 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f874338-aa5e-4a84-b666-8eefe8d2b424 None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:29:07.249790 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f874338-aa5e-4a84-b666-8eefe8d2b424 None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:29:07.250065 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f874338-aa5e-4a84-b666-8eefe8d2b424 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:29:07.260464 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f874338-aa5e-4a84-b666-8eefe8d2b424 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 01 16:29:07.268985 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application [None req-8f874338-aa5e-4a84-b666-8eefe8d2b424 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 01 16:29:07.268985 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 16:29:07.268985 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 16:29:07.268985 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 16:29:07.268985 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 16:29:07.268985 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 16:29:07.268985 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 16:29:07.268985 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 16:29:07.268985 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 16:29:07.268985 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 16:29:07.268985 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 16:29:07.268985 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 16:29:07.268985 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 01 16:29:07.268985 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 01 16:29:07.268985 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 01 16:29:07.268985 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 01 16:29:07.270085 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 01 16:29:07.270085 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 01 16:29:07.270085 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 01 16:29:07.270085 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 01 16:29:07.270085 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 16:29:07.270085 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 16:29:07.270085 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 16:29:07.270085 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 16:29:07.270085 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 16:29:07.270085 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 16:29:07.270085 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 16:29:07.270085 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application with Lock( Jul 01 16:29:07.270085 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 16:29:07.270085 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self._enter() Jul 01 16:29:07.270085 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 16:29:07.270085 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 16:29:07.271070 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 16:29:07.271070 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.creator() Jul 01 16:29:07.271070 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 16:29:07.271070 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application created_value = creator( Jul 01 16:29:07.271070 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 01 16:29:07.271070 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 01 16:29:07.271070 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 01 16:29:07.271070 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 01 16:29:07.271070 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 01 16:29:07.271070 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 01 16:29:07.271070 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 01 16:29:07.271070 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application Jul 01 16:29:07.271070 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 581/1149] 10.222.0.57 () {56 vars in 1096 bytes} [Fri Jul 1 16:29:07 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 16:29:07.276546 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6bad167-cfde-4ae9-b9d6-14a6b77e8e32 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:29:07.374070 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6bad167-cfde-4ae9-b9d6-14a6b77e8e32 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:29:07.390612 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:29:07.409912 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa4a398-e98c-4671-9794-b5b282966aae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:29:07.410786 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa4a398-e98c-4671-9794-b5b282966aae None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:29:07.411255 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa4a398-e98c-4671-9794-b5b282966aae None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:29:07.411750 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa4a398-e98c-4671-9794-b5b282966aae None admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:29:07.413091 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa4a398-e98c-4671-9794-b5b282966aae None admin] RBAC: Adding query filter params (name=admin) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:29:07.413571 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa4a398-e98c-4671-9794-b5b282966aae None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:29:07.415508 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa4a398-e98c-4671-9794-b5b282966aae None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:29:07.424399 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 569/1150] 10.222.0.57 () {56 vars in 1099 bytes} [Fri Jul 1 16:29:07 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:29:07.430498 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f874338-aa5e-4a84-b666-8eefe8d2b424 None admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:29:07.493343 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=100 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:29:07.531680 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f874338-aa5e-4a84-b666-8eefe8d2b424 None admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:29:07.565444 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9145dc0-9c00-47dc-a8e8-0eb6e1166645 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:29:07.566308 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9145dc0-9c00-47dc-a8e8-0eb6e1166645 None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:29:07.566654 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9145dc0-9c00-47dc-a8e8-0eb6e1166645 None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:29:07.566932 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9145dc0-9c00-47dc-a8e8-0eb6e1166645 None admin] PATH_INFO: `/v3/users/f3e86fdc531542f1968af23b4ecac21b` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:29:07.568104 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9145dc0-9c00-47dc-a8e8-0eb6e1166645 None admin] RBAC: Authorizing `identity:get_user(user_id=f3e86fdc531542f1968af23b4ecac21b)` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:29:07.569531 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9145dc0-9c00-47dc-a8e8-0eb6e1166645 None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:29:07.571184 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 582/1151] 10.222.0.57 () {56 vars in 1177 bytes} [Fri Jul 1 16:29:07 2022] GET /identity/v3/users/f3e86fdc531542f1968af23b4ecac21b => generated 286 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:29:07.577383 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa4a398-e98c-4671-9794-b5b282966aae None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:29:07.675333 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa4a398-e98c-4671-9794-b5b282966aae None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:29:07.707880 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2042fbe4-b786-47b5-a7d4-4002179b5caa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:29:07.708735 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2042fbe4-b786-47b5-a7d4-4002179b5caa None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:29:07.709018 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2042fbe4-b786-47b5-a7d4-4002179b5caa None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:29:07.709293 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2042fbe4-b786-47b5-a7d4-4002179b5caa None admin] PATH_INFO: `/v3/projects/2754a147cd5a4b40944818e7ff9bb837` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:29:07.710495 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2042fbe4-b786-47b5-a7d4-4002179b5caa None admin] RBAC: Authorizing `identity:get_project(project_id=2754a147cd5a4b40944818e7ff9bb837)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:29:07.711918 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2042fbe4-b786-47b5-a7d4-4002179b5caa None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:29:07.713625 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 570/1152] 10.222.0.57 () {56 vars in 1186 bytes} [Fri Jul 1 16:29:07 2022] GET /identity/v3/projects/2754a147cd5a4b40944818e7ff9bb837 => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 01 16:29:07.719549 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9145dc0-9c00-47dc-a8e8-0eb6e1166645 None admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:29:07.819307 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9145dc0-9c00-47dc-a8e8-0eb6e1166645 None admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:29:07.849150 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=100 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:29:07.854763 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f877614a-4399-402a-922b-db1715f81161 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:29:07.855657 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f877614a-4399-402a-922b-db1715f81161 None admin] REQUEST_METHOD: `PUT` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:29:07.855943 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f877614a-4399-402a-922b-db1715f81161 None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:29:07.856221 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f877614a-4399-402a-922b-db1715f81161 None admin] PATH_INFO: `/v3/projects/2754a147cd5a4b40944818e7ff9bb837/users/f3e86fdc531542f1968af23b4ecac21b/roles/84ee739146934c1ab32c501ff8093644` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:29:07.858546 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f877614a-4399-402a-922b-db1715f81161 None admin] RBAC: Authorizing `identity:create_grant(project_id=2754a147cd5a4b40944818e7ff9bb837, user_id=f3e86fdc531542f1968af23b4ecac21b, role_id=84ee739146934c1ab32c501ff8093644)` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:29:07.859987 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f877614a-4399-402a-922b-db1715f81161 None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:29:07.873004 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 583/1153] 10.222.0.57 () {58 vars in 1439 bytes} [Fri Jul 1 16:29:07 2022] PUT /identity/v3/projects/2754a147cd5a4b40944818e7ff9bb837/users/f3e86fdc531542f1968af23b4ecac21b/roles/84ee739146934c1ab32c501ff8093644 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 16:29:09.905858 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5a9b22-7bc2-4595-94aa-5bbe970d796f None None] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:29:09.906377 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5a9b22-7bc2-4595-94aa-5bbe970d796f None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:29:09.906585 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5a9b22-7bc2-4595-94aa-5bbe970d796f None None] PATH_INFO: `/` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:29:09.907978 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 571/1154] 10.222.0.57 () {54 vars in 926 bytes} [Fri Jul 1 16:29:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 16:29:09.913215 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0ad864-9080-4fbb-b389-bf0c1a290b07 None None] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:29:09.913617 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0ad864-9080-4fbb-b389-bf0c1a290b07 None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:29:09.913787 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0ad864-9080-4fbb-b389-bf0c1a290b07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:29:09.930825 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.auth.core [None req-3d0ad864-9080-4fbb-b389-bf0c1a290b07 None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80312) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:29:09.932020 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-3d0ad864-9080-4fbb-b389-bf0c1a290b07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:29:10.014829 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 584/1155] 10.222.0.57 () {58 vars in 1023 bytes} [Fri Jul 1 16:29:09 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 01 16:29:10.039277 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5a9b22-7bc2-4595-94aa-5bbe970d796f None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:29:10.086530 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:29:10.142780 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5a9b22-7bc2-4595-94aa-5bbe970d796f None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:29:10.176297 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54dd7c6-bff5-4baf-8f89-5883dcaaac4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:29:10.177065 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54dd7c6-bff5-4baf-8f89-5883dcaaac4b None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:29:10.177348 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54dd7c6-bff5-4baf-8f89-5883dcaaac4b None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:29:10.177625 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54dd7c6-bff5-4baf-8f89-5883dcaaac4b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:29:10.188785 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54dd7c6-bff5-4baf-8f89-5883dcaaac4b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 01 16:29:10.196341 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application [None req-f54dd7c6-bff5-4baf-8f89-5883dcaaac4b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 01 16:29:10.196341 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 16:29:10.196341 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 16:29:10.196341 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 16:29:10.196341 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 16:29:10.196341 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 16:29:10.196341 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 16:29:10.196341 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 16:29:10.196341 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 16:29:10.196341 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 16:29:10.196341 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 16:29:10.196341 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 16:29:10.196341 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 01 16:29:10.196341 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 01 16:29:10.196341 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 01 16:29:10.196341 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 01 16:29:10.197467 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 01 16:29:10.197467 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 01 16:29:10.197467 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 01 16:29:10.197467 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 01 16:29:10.197467 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 16:29:10.197467 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 16:29:10.197467 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 16:29:10.197467 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 16:29:10.197467 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 16:29:10.197467 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 16:29:10.197467 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 16:29:10.197467 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application with Lock( Jul 01 16:29:10.197467 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 16:29:10.197467 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._enter() Jul 01 16:29:10.197467 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 16:29:10.197467 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 16:29:10.203424 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 16:29:10.203424 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.creator() Jul 01 16:29:10.203424 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 16:29:10.203424 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application created_value = creator( Jul 01 16:29:10.203424 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 01 16:29:10.203424 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 01 16:29:10.203424 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 01 16:29:10.203424 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 01 16:29:10.203424 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 01 16:29:10.203424 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 01 16:29:10.203424 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 01 16:29:10.203424 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Jul 01 16:29:10.203424 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 572/1156] 10.222.0.57 () {56 vars in 1096 bytes} [Fri Jul 1 16:29:10 2022] GET /identity/v3/roles/admin => generated 83 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 16:29:10.204295 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0ad864-9080-4fbb-b389-bf0c1a290b07 None None] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:29:10.301102 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0ad864-9080-4fbb-b389-bf0c1a290b07 None None] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:29:10.323217 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:29:10.336977 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a86675-3ce6-48b0-8de4-134859f4cd37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:29:10.337655 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a86675-3ce6-48b0-8de4-134859f4cd37 None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:29:10.337942 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a86675-3ce6-48b0-8de4-134859f4cd37 None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:29:10.338225 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a86675-3ce6-48b0-8de4-134859f4cd37 None admin] PATH_INFO: `/v3/roles` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:29:10.339009 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a86675-3ce6-48b0-8de4-134859f4cd37 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80312) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:29:10.339321 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a86675-3ce6-48b0-8de4-134859f4cd37 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:29:10.340697 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a86675-3ce6-48b0-8de4-134859f4cd37 None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:29:10.349575 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 585/1157] 10.222.0.57 () {56 vars in 1099 bytes} [Fri Jul 1 16:29:10 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:29:10.355388 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54dd7c6-bff5-4baf-8f89-5883dcaaac4b None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:29:10.454178 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54dd7c6-bff5-4baf-8f89-5883dcaaac4b None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:29:10.460933 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:29:10.487608 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7edfdd66-698d-40cb-9e3f-fe233357c7ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:29:10.488451 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edfdd66-698d-40cb-9e3f-fe233357c7ae None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:29:10.488733 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edfdd66-698d-40cb-9e3f-fe233357c7ae None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:29:10.489007 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edfdd66-698d-40cb-9e3f-fe233357c7ae None admin] PATH_INFO: `/v3/users/f3e86fdc531542f1968af23b4ecac21b` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:29:10.490163 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7edfdd66-698d-40cb-9e3f-fe233357c7ae None admin] RBAC: Authorizing `identity:get_user(user_id=f3e86fdc531542f1968af23b4ecac21b)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:29:10.491619 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7edfdd66-698d-40cb-9e3f-fe233357c7ae None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:29:10.493226 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 573/1158] 10.222.0.57 () {56 vars in 1177 bytes} [Fri Jul 1 16:29:10 2022] GET /identity/v3/users/f3e86fdc531542f1968af23b4ecac21b => generated 286 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:29:10.498998 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a86675-3ce6-48b0-8de4-134859f4cd37 None admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:29:10.600370 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a86675-3ce6-48b0-8de4-134859f4cd37 None admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:29:10.632600 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae9133c-befe-41cd-a14c-2008dabcc046 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:29:10.633240 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae9133c-befe-41cd-a14c-2008dabcc046 None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:29:10.633527 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae9133c-befe-41cd-a14c-2008dabcc046 None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:29:10.633809 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae9133c-befe-41cd-a14c-2008dabcc046 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:29:10.683942 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=100 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:29:10.700938 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae9133c-befe-41cd-a14c-2008dabcc046 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:29:10.702673 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae9133c-befe-41cd-a14c-2008dabcc046 None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:29:10.797855 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 586/1159] 10.222.0.57 () {58 vars in 1282 bytes} [Fri Jul 1 16:29:10 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 01 16:29:10.803718 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7edfdd66-698d-40cb-9e3f-fe233357c7ae None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:29:10.901561 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7edfdd66-698d-40cb-9e3f-fe233357c7ae None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:29:10.933481 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f4ee37-2688-4c33-9fd0-cf818732ee6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:29:10.934324 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f4ee37-2688-4c33-9fd0-cf818732ee6c None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:29:10.934634 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f4ee37-2688-4c33-9fd0-cf818732ee6c None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:29:10.934906 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f4ee37-2688-4c33-9fd0-cf818732ee6c None admin] PATH_INFO: `/v3/projects/2754a147cd5a4b40944818e7ff9bb837` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:29:10.936058 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f4ee37-2688-4c33-9fd0-cf818732ee6c None admin] RBAC: Authorizing `identity:get_project(project_id=2754a147cd5a4b40944818e7ff9bb837)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:29:10.937463 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f4ee37-2688-4c33-9fd0-cf818732ee6c None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:29:10.939899 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 574/1160] 10.222.0.57 () {56 vars in 1186 bytes} [Fri Jul 1 16:29:10 2022] GET /identity/v3/projects/2754a147cd5a4b40944818e7ff9bb837 => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:29:10.945953 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae9133c-befe-41cd-a14c-2008dabcc046 None admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:29:11.026697 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=100 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:29:11.042621 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae9133c-befe-41cd-a14c-2008dabcc046 None admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:29:11.075355 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82da41f5-aeb0-4a9b-b473-2531072152be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:29:11.075933 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82da41f5-aeb0-4a9b-b473-2531072152be None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:29:11.076465 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82da41f5-aeb0-4a9b-b473-2531072152be None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:29:11.076742 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82da41f5-aeb0-4a9b-b473-2531072152be None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:29:11.077557 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82da41f5-aeb0-4a9b-b473-2531072152be None admin] RBAC: Adding query filter params (role.id=84ee739146934c1ab32c501ff8093644, scope.project.id=2754a147cd5a4b40944818e7ff9bb837, user.id=f3e86fdc531542f1968af23b4ecac21b) {{(pid=80312) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:29:11.077866 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82da41f5-aeb0-4a9b-b473-2531072152be None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:29:11.079464 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82da41f5-aeb0-4a9b-b473-2531072152be None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:29:11.085090 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 587/1161] 10.222.0.57 () {56 vars in 1374 bytes} [Fri Jul 1 16:29:10 2022] GET /identity/v3/role_assignments?user.id=f3e86fdc531542f1968af23b4ecac21b&scope.project.id=2754a147cd5a4b40944818e7ff9bb837&role.id=84ee739146934c1ab32c501ff8093644 => generated 608 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:29:13.166999 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29987cea-fda3-4a9d-a7e5-4aac870ba401 None None] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:29:13.167549 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29987cea-fda3-4a9d-a7e5-4aac870ba401 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:29:13.167549 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29987cea-fda3-4a9d-a7e5-4aac870ba401 None None] PATH_INFO: `/` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:29:13.168862 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 575/1162] 10.222.0.57 () {54 vars in 926 bytes} [Fri Jul 1 16:29:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 16:29:13.174651 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8588ce78-482b-4450-bc31-2c99eb39f05f None None] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:29:13.175150 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8588ce78-482b-4450-bc31-2c99eb39f05f None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:29:13.175259 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8588ce78-482b-4450-bc31-2c99eb39f05f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:29:13.192091 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.auth.core [None req-8588ce78-482b-4450-bc31-2c99eb39f05f None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80312) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:29:13.193278 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-8588ce78-482b-4450-bc31-2c99eb39f05f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:29:13.259563 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 588/1163] 10.222.0.57 () {58 vars in 1023 bytes} [Fri Jul 1 16:29:13 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 01 16:29:13.282655 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29987cea-fda3-4a9d-a7e5-4aac870ba401 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:29:13.332114 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:29:13.388026 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29987cea-fda3-4a9d-a7e5-4aac870ba401 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:29:13.427570 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2c30ae-095a-4acd-9bea-7b710fcec525 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:29:13.428152 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2c30ae-095a-4acd-9bea-7b710fcec525 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:29:13.428437 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2c30ae-095a-4acd-9bea-7b710fcec525 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:29:13.428716 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2c30ae-095a-4acd-9bea-7b710fcec525 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:29:13.493858 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b2c30ae-095a-4acd-9bea-7b710fcec525 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:29:13.495319 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b2c30ae-095a-4acd-9bea-7b710fcec525 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:29:13.538078 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:29:13.594296 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 576/1164] 10.222.0.57 () {58 vars in 1282 bytes} [Fri Jul 1 16:29:13 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 01 16:29:13.600248 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8588ce78-482b-4450-bc31-2c99eb39f05f None None] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:29:13.677000 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=100 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:29:13.698841 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8588ce78-482b-4450-bc31-2c99eb39f05f None None] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:29:13.731571 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976f1d6e-c193-4dd1-8f05-cca5f9fe6f01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:29:13.732340 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976f1d6e-c193-4dd1-8f05-cca5f9fe6f01 None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:29:13.732622 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976f1d6e-c193-4dd1-8f05-cca5f9fe6f01 None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:29:13.732901 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976f1d6e-c193-4dd1-8f05-cca5f9fe6f01 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:29:13.739971 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-976f1d6e-c193-4dd1-8f05-cca5f9fe6f01 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:29:13.741399 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-976f1d6e-c193-4dd1-8f05-cca5f9fe6f01 None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:29:13.755872 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application [None req-976f1d6e-c193-4dd1-8f05-cca5f9fe6f01 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Jul 01 16:29:13.755872 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 16:29:13.755872 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Jul 01 16:29:13.755872 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Jul 01 16:29:13.755872 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 01 16:29:13.755872 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 01 16:29:13.755872 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 01 16:29:13.755872 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 01 16:29:13.755872 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Jul 01 16:29:13.755872 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application Jul 01 16:29:13.755872 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 01 16:29:13.755872 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application Jul 01 16:29:13.755872 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 16:29:13.755872 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 16:29:13.755872 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 16:29:13.755872 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 16:29:13.755872 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 16:29:13.757165 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 16:29:13.757165 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 16:29:13.757165 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 16:29:13.757165 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 16:29:13.757165 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 16:29:13.757165 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 16:29:13.757165 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 01 16:29:13.757165 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 01 16:29:13.757165 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 01 16:29:13.757165 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 01 16:29:13.757165 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 16:29:13.757165 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 16:29:13.757165 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 16:29:13.757165 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 16:29:13.757165 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 16:29:13.757165 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 16:29:13.758180 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 16:29:13.758180 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application with Lock( Jul 01 16:29:13.758180 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 16:29:13.758180 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self._enter() Jul 01 16:29:13.758180 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 16:29:13.758180 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 16:29:13.758180 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 16:29:13.758180 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.creator() Jul 01 16:29:13.758180 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 16:29:13.758180 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application created_value = creator( Jul 01 16:29:13.758180 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Jul 01 16:29:13.758180 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 01 16:29:13.758180 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Jul 01 16:29:13.758180 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application Jul 01 16:29:13.758180 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 589/1165] 10.222.0.57 () {56 vars in 1117 bytes} [Fri Jul 1 16:29:13 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 16:29:13.763071 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2c30ae-095a-4acd-9bea-7b710fcec525 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:29:13.861584 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2c30ae-095a-4acd-9bea-7b710fcec525 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:29:13.894980 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3dfaa0a-2efe-4eed-a60c-2c0f5fd33ebc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:29:13.895625 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3dfaa0a-2efe-4eed-a60c-2c0f5fd33ebc None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:29:13.895904 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3dfaa0a-2efe-4eed-a60c-2c0f5fd33ebc None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:29:13.896176 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3dfaa0a-2efe-4eed-a60c-2c0f5fd33ebc None admin] PATH_INFO: `/v3/domains` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:29:13.896886 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3dfaa0a-2efe-4eed-a60c-2c0f5fd33ebc None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:29:13.897187 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3dfaa0a-2efe-4eed-a60c-2c0f5fd33ebc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:29:13.898555 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3dfaa0a-2efe-4eed-a60c-2c0f5fd33ebc None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:29:13.905457 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 577/1166] 10.222.0.57 () {56 vars in 1115 bytes} [Fri Jul 1 16:29:13 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:29:15.937226 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509201a1-fbc0-4dc4-ac28-c1440db6b14c None None] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:29:15.937797 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509201a1-fbc0-4dc4-ac28-c1440db6b14c None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:29:15.937904 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509201a1-fbc0-4dc4-ac28-c1440db6b14c None None] PATH_INFO: `/` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:29:15.939281 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 590/1167] 10.222.0.57 () {54 vars in 926 bytes} [Fri Jul 1 16:29:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 16:29:15.944500 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3848ed6d-ed5c-49f9-9e3d-354309a69910 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:29:15.944893 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3848ed6d-ed5c-49f9-9e3d-354309a69910 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:29:15.945063 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3848ed6d-ed5c-49f9-9e3d-354309a69910 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:29:15.957402 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:29:15.962751 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-3848ed6d-ed5c-49f9-9e3d-354309a69910 None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:29:15.963950 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-3848ed6d-ed5c-49f9-9e3d-354309a69910 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:29:16.030127 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 578/1168] 10.222.0.57 () {58 vars in 1023 bytes} [Fri Jul 1 16:29:15 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 01 16:29:16.053475 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509201a1-fbc0-4dc4-ac28-c1440db6b14c None None] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:29:16.154322 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509201a1-fbc0-4dc4-ac28-c1440db6b14c None None] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:29:16.187182 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d1e626-4811-4ab3-b3c2-196ab0481f84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:29:16.188391 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d1e626-4811-4ab3-b3c2-196ab0481f84 None admin] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:29:16.188677 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d1e626-4811-4ab3-b3c2-196ab0481f84 None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:29:16.188953 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d1e626-4811-4ab3-b3c2-196ab0481f84 None admin] PATH_INFO: `/v3/domains` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:29:16.189694 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d1e626-4811-4ab3-b3c2-196ab0481f84 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:29:16.191078 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d1e626-4811-4ab3-b3c2-196ab0481f84 None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:29:16.199843 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:29:16.208395 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 591/1169] 10.222.0.57 () {60 vars in 1138 bytes} [Fri Jul 1 16:29:16 2022] POST /identity/v3/domains => generated 267 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 16:29:18.285184 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6dc87d6-38e7-45d3-811d-c6e4e9b5d462 None None] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:29:18.285724 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6dc87d6-38e7-45d3-811d-c6e4e9b5d462 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:29:18.285724 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6dc87d6-38e7-45d3-811d-c6e4e9b5d462 None None] PATH_INFO: `/` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:29:18.287229 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 579/1170] 10.222.0.57 () {54 vars in 926 bytes} [Fri Jul 1 16:29:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 16:29:18.293062 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de45e8fd-cb8d-4e1c-9090-d57f7ed5f4b8 None None] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:29:18.293481 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de45e8fd-cb8d-4e1c-9090-d57f7ed5f4b8 None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:29:18.293629 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de45e8fd-cb8d-4e1c-9090-d57f7ed5f4b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:29:18.310854 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.auth.core [None req-de45e8fd-cb8d-4e1c-9090-d57f7ed5f4b8 None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80312) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:29:18.312048 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-de45e8fd-cb8d-4e1c-9090-d57f7ed5f4b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:29:18.378006 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 592/1171] 10.222.0.57 () {58 vars in 1023 bytes} [Fri Jul 1 16:29:18 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 01 16:29:18.401734 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6dc87d6-38e7-45d3-811d-c6e4e9b5d462 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:29:18.501062 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6dc87d6-38e7-45d3-811d-c6e4e9b5d462 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:29:18.535298 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb1ea8e-56e4-43e0-9de7-ff695eda3c85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:29:18.536068 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb1ea8e-56e4-43e0-9de7-ff695eda3c85 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:29:18.536346 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb1ea8e-56e4-43e0-9de7-ff695eda3c85 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:29:18.536617 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb1ea8e-56e4-43e0-9de7-ff695eda3c85 None admin] PATH_INFO: `/v3/domains/650f8a19fff44cd4988380e6d1dbccf2` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:29:18.539272 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:29:18.547912 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb1ea8e-56e4-43e0-9de7-ff695eda3c85 None admin] RBAC: Authorizing `identity:get_domain(domain_id=650f8a19fff44cd4988380e6d1dbccf2)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:29:18.549348 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb1ea8e-56e4-43e0-9de7-ff695eda3c85 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:29:18.551011 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 580/1172] 10.222.0.57 () {56 vars in 1183 bytes} [Fri Jul 1 16:29:18 2022] GET /identity/v3/domains/650f8a19fff44cd4988380e6d1dbccf2 => generated 267 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:29:18.556889 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de45e8fd-cb8d-4e1c-9090-d57f7ed5f4b8 None None] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:29:18.653437 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de45e8fd-cb8d-4e1c-9090-d57f7ed5f4b8 None None] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:29:18.679066 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=100 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:29:18.687100 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b417de3-ea39-4d5e-b13f-3c09b5c14df8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:29:18.687817 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b417de3-ea39-4d5e-b13f-3c09b5c14df8 None admin] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:29:18.688102 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b417de3-ea39-4d5e-b13f-3c09b5c14df8 None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:29:18.688411 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b417de3-ea39-4d5e-b13f-3c09b5c14df8 None admin] PATH_INFO: `/v3/projects` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:29:18.689116 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b417de3-ea39-4d5e-b13f-3c09b5c14df8 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:29:18.690580 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b417de3-ea39-4d5e-b13f-3c09b5c14df8 None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:29:18.715067 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 593/1173] 10.222.0.57 () {60 vars in 1141 bytes} [Fri Jul 1 16:29:18 2022] POST /identity/v3/projects => generated 361 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 16:29:20.846465 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cc79bb-0fd5-4362-84cd-3a33960ff7a7 None None] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:29:20.847030 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cc79bb-0fd5-4362-84cd-3a33960ff7a7 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:29:20.847030 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cc79bb-0fd5-4362-84cd-3a33960ff7a7 None None] PATH_INFO: `/` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:29:20.848370 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 581/1174] 10.222.0.57 () {54 vars in 926 bytes} [Fri Jul 1 16:29:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 16:29:20.853532 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816b9284-83c6-469c-ab72-d5f0bae468e2 None None] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:29:20.853910 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816b9284-83c6-469c-ab72-d5f0bae468e2 None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:29:20.854088 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816b9284-83c6-469c-ab72-d5f0bae468e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:29:20.871018 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.auth.core [None req-816b9284-83c6-469c-ab72-d5f0bae468e2 None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80312) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:29:20.872197 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-816b9284-83c6-469c-ab72-d5f0bae468e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:29:20.953966 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 594/1175] 10.222.0.57 () {58 vars in 1023 bytes} [Fri Jul 1 16:29:20 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 01 16:29:20.983237 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3cc79bb-0fd5-4362-84cd-3a33960ff7a7 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:29:21.084289 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3cc79bb-0fd5-4362-84cd-3a33960ff7a7 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:29:21.117624 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb29ccc-bc24-49a1-9afe-1ff200fdb19c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:29:21.118438 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb29ccc-bc24-49a1-9afe-1ff200fdb19c None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:29:21.118726 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb29ccc-bc24-49a1-9afe-1ff200fdb19c None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:29:21.119001 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb29ccc-bc24-49a1-9afe-1ff200fdb19c None admin] PATH_INFO: `/v3/domains/650f8a19fff44cd4988380e6d1dbccf2` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:29:21.120183 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb29ccc-bc24-49a1-9afe-1ff200fdb19c None admin] RBAC: Authorizing `identity:get_domain(domain_id=650f8a19fff44cd4988380e6d1dbccf2)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:29:21.121613 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb29ccc-bc24-49a1-9afe-1ff200fdb19c None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:29:21.123267 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 582/1176] 10.222.0.57 () {56 vars in 1183 bytes} [Fri Jul 1 16:29:20 2022] GET /identity/v3/domains/650f8a19fff44cd4988380e6d1dbccf2 => generated 267 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:29:21.129883 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816b9284-83c6-469c-ab72-d5f0bae468e2 None None] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:29:21.225156 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816b9284-83c6-469c-ab72-d5f0bae468e2 None None] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:29:21.237865 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:29:21.263080 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faed6afb-4002-422c-b55e-6ca75d87348a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:29:21.263767 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faed6afb-4002-422c-b55e-6ca75d87348a None admin] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:29:21.264052 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faed6afb-4002-422c-b55e-6ca75d87348a None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:29:21.264332 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faed6afb-4002-422c-b55e-6ca75d87348a None admin] PATH_INFO: `/v3/users` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:29:21.265062 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faed6afb-4002-422c-b55e-6ca75d87348a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:29:21.266586 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faed6afb-4002-422c-b55e-6ca75d87348a None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:29:21.287264 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 595/1177] 10.222.0.57 () {60 vars in 1132 bytes} [Fri Jul 1 16:29:21 2022] POST /identity/v3/users => generated 311 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 16:29:23.381055 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91854a01-8669-4463-b265-ceb30f69430f None None] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:29:23.381610 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91854a01-8669-4463-b265-ceb30f69430f None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:29:23.381610 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91854a01-8669-4463-b265-ceb30f69430f None None] PATH_INFO: `/` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:29:23.382997 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 583/1178] 10.222.0.57 () {54 vars in 926 bytes} [Fri Jul 1 16:29:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 16:29:23.388205 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8457812-449f-4b95-8674-26280c808842 None None] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:29:23.388614 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8457812-449f-4b95-8674-26280c808842 None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:29:23.388773 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8457812-449f-4b95-8674-26280c808842 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:29:23.405767 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.auth.core [None req-d8457812-449f-4b95-8674-26280c808842 None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80312) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:29:23.407552 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-d8457812-449f-4b95-8674-26280c808842 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:29:23.472932 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 596/1179] 10.222.0.57 () {58 vars in 1023 bytes} [Fri Jul 1 16:29:23 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 01 16:29:23.496589 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91854a01-8669-4463-b265-ceb30f69430f None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:29:23.559294 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:29:23.597843 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91854a01-8669-4463-b265-ceb30f69430f None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:29:23.630735 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671e6ab4-0f24-4dfa-93a4-65bea8ff131f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:29:23.631655 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671e6ab4-0f24-4dfa-93a4-65bea8ff131f None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:29:23.631986 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671e6ab4-0f24-4dfa-93a4-65bea8ff131f None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:29:23.632307 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671e6ab4-0f24-4dfa-93a4-65bea8ff131f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:29:23.643338 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-671e6ab4-0f24-4dfa-93a4-65bea8ff131f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 01 16:29:23.650829 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application [None req-671e6ab4-0f24-4dfa-93a4-65bea8ff131f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 01 16:29:23.650829 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 16:29:23.650829 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 16:29:23.650829 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 16:29:23.650829 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 16:29:23.650829 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 16:29:23.650829 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 16:29:23.650829 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 16:29:23.650829 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 16:29:23.650829 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 16:29:23.650829 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 16:29:23.650829 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 16:29:23.650829 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 01 16:29:23.650829 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 01 16:29:23.650829 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 01 16:29:23.650829 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 01 16:29:23.651912 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 01 16:29:23.651912 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 01 16:29:23.651912 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 01 16:29:23.651912 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 01 16:29:23.651912 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 16:29:23.651912 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 16:29:23.651912 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 16:29:23.651912 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 16:29:23.651912 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 16:29:23.651912 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 16:29:23.651912 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 16:29:23.651912 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application with Lock( Jul 01 16:29:23.651912 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 16:29:23.651912 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._enter() Jul 01 16:29:23.651912 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 16:29:23.651912 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 16:29:23.653014 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 16:29:23.653014 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.creator() Jul 01 16:29:23.653014 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 16:29:23.653014 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application created_value = creator( Jul 01 16:29:23.653014 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 01 16:29:23.653014 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 01 16:29:23.653014 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 01 16:29:23.653014 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 01 16:29:23.653014 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 01 16:29:23.653014 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 01 16:29:23.653014 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 01 16:29:23.653014 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Jul 01 16:29:23.653014 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 584/1180] 10.222.0.57 () {56 vars in 1096 bytes} [Fri Jul 1 16:29:23 2022] GET /identity/v3/roles/admin => generated 83 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 16:29:23.658268 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8457812-449f-4b95-8674-26280c808842 None None] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:29:23.749193 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:29:23.757820 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8457812-449f-4b95-8674-26280c808842 None None] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:29:23.795398 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f9671a-66c7-4915-ab7e-b75383f75dbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:29:23.796065 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f9671a-66c7-4915-ab7e-b75383f75dbe None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:29:23.796345 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f9671a-66c7-4915-ab7e-b75383f75dbe None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:29:23.796759 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f9671a-66c7-4915-ab7e-b75383f75dbe None admin] PATH_INFO: `/v3/roles` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:29:23.797672 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f9671a-66c7-4915-ab7e-b75383f75dbe None admin] RBAC: Adding query filter params (name=admin) {{(pid=80312) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:29:23.797981 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f9671a-66c7-4915-ab7e-b75383f75dbe None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:29:23.799377 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f9671a-66c7-4915-ab7e-b75383f75dbe None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:29:23.807244 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 597/1181] 10.222.0.57 () {56 vars in 1099 bytes} [Fri Jul 1 16:29:23 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:29:23.813733 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671e6ab4-0f24-4dfa-93a4-65bea8ff131f None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:29:23.923183 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671e6ab4-0f24-4dfa-93a4-65bea8ff131f None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:29:23.951900 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:29:23.964063 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f1fd4d-034b-469a-94ab-cafbbbf568cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:29:23.964939 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f1fd4d-034b-469a-94ab-cafbbbf568cc None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:29:23.965217 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f1fd4d-034b-469a-94ab-cafbbbf568cc None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:29:23.965493 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f1fd4d-034b-469a-94ab-cafbbbf568cc None admin] PATH_INFO: `/v3/users/e1ff4cbdd194489a88a4fb4432312d5f` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:29:23.980346 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f1fd4d-034b-469a-94ab-cafbbbf568cc None admin] RBAC: Authorizing `identity:get_user(user_id=e1ff4cbdd194489a88a4fb4432312d5f)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:29:23.981799 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f1fd4d-034b-469a-94ab-cafbbbf568cc None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:29:23.983559 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 585/1182] 10.222.0.57 () {56 vars in 1177 bytes} [Fri Jul 1 16:29:23 2022] GET /identity/v3/users/e1ff4cbdd194489a88a4fb4432312d5f => generated 311 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:29:23.990930 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f9671a-66c7-4915-ab7e-b75383f75dbe None admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:29:24.099715 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f9671a-66c7-4915-ab7e-b75383f75dbe None admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:29:24.134769 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483a4816-78e7-4a1b-af66-a3ac58d68cec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:29:24.135423 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483a4816-78e7-4a1b-af66-a3ac58d68cec None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:29:24.135710 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483a4816-78e7-4a1b-af66-a3ac58d68cec None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:29:24.135991 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483a4816-78e7-4a1b-af66-a3ac58d68cec None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:29:24.163871 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=100 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:29:24.208646 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-483a4816-78e7-4a1b-af66-a3ac58d68cec None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:29:24.210378 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-483a4816-78e7-4a1b-af66-a3ac58d68cec None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:29:24.326202 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 598/1183] 10.222.0.57 () {58 vars in 1282 bytes} [Fri Jul 1 16:29:23 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 01 16:29:24.332372 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f1fd4d-034b-469a-94ab-cafbbbf568cc None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:29:24.448651 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f1fd4d-034b-469a-94ab-cafbbbf568cc None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:29:24.489219 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e39a50-4017-4205-9eab-057502722f3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:29:24.490000 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e39a50-4017-4205-9eab-057502722f3b None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:29:24.490288 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e39a50-4017-4205-9eab-057502722f3b None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:29:24.490590 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e39a50-4017-4205-9eab-057502722f3b None admin] PATH_INFO: `/v3/projects/f4db6b15edb4481f9ccc8f08c45b179b` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:29:24.491788 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e39a50-4017-4205-9eab-057502722f3b None admin] RBAC: Authorizing `identity:get_project(project_id=f4db6b15edb4481f9ccc8f08c45b179b)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:29:24.493228 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e39a50-4017-4205-9eab-057502722f3b None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:29:24.494985 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 586/1184] 10.222.0.57 () {56 vars in 1186 bytes} [Fri Jul 1 16:29:24 2022] GET /identity/v3/projects/f4db6b15edb4481f9ccc8f08c45b179b => generated 361 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:29:24.501321 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483a4816-78e7-4a1b-af66-a3ac58d68cec None admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:29:24.567108 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=100 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:29:24.608697 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483a4816-78e7-4a1b-af66-a3ac58d68cec None admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:29:24.642221 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba7f648-fe0f-4fd2-b686-8fa205071452 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:29:24.642894 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba7f648-fe0f-4fd2-b686-8fa205071452 None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:29:24.643181 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba7f648-fe0f-4fd2-b686-8fa205071452 None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:29:24.643457 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba7f648-fe0f-4fd2-b686-8fa205071452 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:29:24.644280 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba7f648-fe0f-4fd2-b686-8fa205071452 None admin] RBAC: Adding query filter params (role.id=84ee739146934c1ab32c501ff8093644, scope.project.id=f4db6b15edb4481f9ccc8f08c45b179b, user.id=e1ff4cbdd194489a88a4fb4432312d5f) {{(pid=80312) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:29:24.644594 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba7f648-fe0f-4fd2-b686-8fa205071452 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:29:24.646014 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba7f648-fe0f-4fd2-b686-8fa205071452 None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:29:24.651400 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 599/1185] 10.222.0.57 () {56 vars in 1374 bytes} [Fri Jul 1 16:29:24 2022] GET /identity/v3/role_assignments?user.id=e1ff4cbdd194489a88a4fb4432312d5f&scope.project.id=f4db6b15edb4481f9ccc8f08c45b179b&role.id=84ee739146934c1ab32c501ff8093644 => generated 259 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:29:26.641464 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62976424-33f7-4c7d-ba98-6cbae2203686 None None] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:29:26.642006 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62976424-33f7-4c7d-ba98-6cbae2203686 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:29:26.642006 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62976424-33f7-4c7d-ba98-6cbae2203686 None None] PATH_INFO: `/` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:29:26.643408 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 587/1186] 10.222.0.57 () {54 vars in 926 bytes} [Fri Jul 1 16:29:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 16:29:26.648541 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6120fa71-b0ad-4337-8df6-19c00b89e656 None None] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:29:26.648957 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6120fa71-b0ad-4337-8df6-19c00b89e656 None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:29:26.649175 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6120fa71-b0ad-4337-8df6-19c00b89e656 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:29:26.666663 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.auth.core [None req-6120fa71-b0ad-4337-8df6-19c00b89e656 None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80312) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:29:26.667836 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-6120fa71-b0ad-4337-8df6-19c00b89e656 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:29:26.733868 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 600/1187] 10.222.0.57 () {58 vars in 1023 bytes} [Fri Jul 1 16:29:26 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 01 16:29:26.756326 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62976424-33f7-4c7d-ba98-6cbae2203686 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:29:26.806665 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:29:26.857359 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62976424-33f7-4c7d-ba98-6cbae2203686 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:29:26.894038 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6258abc7-feba-4fb3-93d3-9a2a11a12635 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:29:26.894768 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6258abc7-feba-4fb3-93d3-9a2a11a12635 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:29:26.895057 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6258abc7-feba-4fb3-93d3-9a2a11a12635 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:29:26.895338 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6258abc7-feba-4fb3-93d3-9a2a11a12635 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:29:26.905206 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6258abc7-feba-4fb3-93d3-9a2a11a12635 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 01 16:29:26.914335 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application [None req-6258abc7-feba-4fb3-93d3-9a2a11a12635 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 01 16:29:26.914335 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 16:29:26.914335 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 16:29:26.914335 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 16:29:26.914335 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 16:29:26.914335 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 16:29:26.914335 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 16:29:26.914335 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 16:29:26.914335 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 16:29:26.914335 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 16:29:26.914335 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 16:29:26.914335 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 16:29:26.914335 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 01 16:29:26.914335 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 01 16:29:26.914335 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 01 16:29:26.914335 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 01 16:29:26.915521 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 01 16:29:26.915521 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 01 16:29:26.915521 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 01 16:29:26.915521 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 01 16:29:26.915521 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 16:29:26.915521 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 16:29:26.915521 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 16:29:26.915521 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 16:29:26.915521 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 16:29:26.915521 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 16:29:26.915521 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 16:29:26.915521 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application with Lock( Jul 01 16:29:26.915521 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 16:29:26.915521 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._enter() Jul 01 16:29:26.915521 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 16:29:26.915521 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 16:29:26.916538 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 16:29:26.916538 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.creator() Jul 01 16:29:26.916538 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 16:29:26.916538 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application created_value = creator( Jul 01 16:29:26.916538 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 01 16:29:26.916538 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 01 16:29:26.916538 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 01 16:29:26.916538 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 01 16:29:26.916538 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 01 16:29:26.916538 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 01 16:29:26.916538 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 01 16:29:26.916538 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Jul 01 16:29:26.917189 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 588/1188] 10.222.0.57 () {56 vars in 1096 bytes} [Fri Jul 1 16:29:26 2022] GET /identity/v3/roles/admin => generated 83 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 16:29:26.925886 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6120fa71-b0ad-4337-8df6-19c00b89e656 None None] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:29:26.977263 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=100 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:29:27.023856 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6120fa71-b0ad-4337-8df6-19c00b89e656 None None] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:29:27.057245 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e54705-c08e-491a-86d8-9db6cad9db37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:29:27.057905 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e54705-c08e-491a-86d8-9db6cad9db37 None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:29:27.058193 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e54705-c08e-491a-86d8-9db6cad9db37 None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:29:27.058502 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e54705-c08e-491a-86d8-9db6cad9db37 None admin] PATH_INFO: `/v3/roles` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:29:27.059249 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e54705-c08e-491a-86d8-9db6cad9db37 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80312) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:29:27.059559 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e54705-c08e-491a-86d8-9db6cad9db37 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:29:27.060952 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e54705-c08e-491a-86d8-9db6cad9db37 None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:29:27.069179 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 601/1189] 10.222.0.57 () {56 vars in 1099 bytes} [Fri Jul 1 16:29:26 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:29:27.075017 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6258abc7-feba-4fb3-93d3-9a2a11a12635 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:29:27.173326 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6258abc7-feba-4fb3-93d3-9a2a11a12635 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:29:27.183188 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:29:27.210596 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eeceb2a-8e6c-4f58-8a71-58af7e7a27d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:29:27.211473 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eeceb2a-8e6c-4f58-8a71-58af7e7a27d3 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:29:27.211759 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eeceb2a-8e6c-4f58-8a71-58af7e7a27d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:29:27.212040 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eeceb2a-8e6c-4f58-8a71-58af7e7a27d3 None admin] PATH_INFO: `/v3/users/e1ff4cbdd194489a88a4fb4432312d5f` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:29:27.213234 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eeceb2a-8e6c-4f58-8a71-58af7e7a27d3 None admin] RBAC: Authorizing `identity:get_user(user_id=e1ff4cbdd194489a88a4fb4432312d5f)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:29:27.214711 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eeceb2a-8e6c-4f58-8a71-58af7e7a27d3 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:29:27.216382 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 589/1190] 10.222.0.57 () {56 vars in 1177 bytes} [Fri Jul 1 16:29:27 2022] GET /identity/v3/users/e1ff4cbdd194489a88a4fb4432312d5f => generated 311 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:29:27.222912 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e54705-c08e-491a-86d8-9db6cad9db37 None admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:29:27.321770 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e54705-c08e-491a-86d8-9db6cad9db37 None admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:29:27.335685 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=100 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:29:27.354994 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35006d2-8af5-44e1-aeaf-51c68a45521a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:29:27.355847 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35006d2-8af5-44e1-aeaf-51c68a45521a None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:29:27.356127 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35006d2-8af5-44e1-aeaf-51c68a45521a None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:29:27.356398 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35006d2-8af5-44e1-aeaf-51c68a45521a None admin] PATH_INFO: `/v3/projects/f4db6b15edb4481f9ccc8f08c45b179b` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:29:27.357550 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a35006d2-8af5-44e1-aeaf-51c68a45521a None admin] RBAC: Authorizing `identity:get_project(project_id=f4db6b15edb4481f9ccc8f08c45b179b)` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:29:27.359203 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a35006d2-8af5-44e1-aeaf-51c68a45521a None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:29:27.360872 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 602/1191] 10.222.0.57 () {56 vars in 1186 bytes} [Fri Jul 1 16:29:27 2022] GET /identity/v3/projects/f4db6b15edb4481f9ccc8f08c45b179b => generated 361 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:29:27.366750 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eeceb2a-8e6c-4f58-8a71-58af7e7a27d3 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:29:27.468317 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eeceb2a-8e6c-4f58-8a71-58af7e7a27d3 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:29:27.500984 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616df789-884e-4a1c-9903-7d943e242274 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:29:27.501857 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616df789-884e-4a1c-9903-7d943e242274 None admin] REQUEST_METHOD: `PUT` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:29:27.502143 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616df789-884e-4a1c-9903-7d943e242274 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:29:27.502450 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616df789-884e-4a1c-9903-7d943e242274 None admin] PATH_INFO: `/v3/projects/f4db6b15edb4481f9ccc8f08c45b179b/users/e1ff4cbdd194489a88a4fb4432312d5f/roles/84ee739146934c1ab32c501ff8093644` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:29:27.504390 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616df789-884e-4a1c-9903-7d943e242274 None admin] RBAC: Authorizing `identity:create_grant(project_id=f4db6b15edb4481f9ccc8f08c45b179b, user_id=e1ff4cbdd194489a88a4fb4432312d5f, role_id=84ee739146934c1ab32c501ff8093644)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:29:27.505817 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616df789-884e-4a1c-9903-7d943e242274 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:29:27.519090 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 590/1192] 10.222.0.57 () {58 vars in 1439 bytes} [Fri Jul 1 16:29:27 2022] PUT /identity/v3/projects/f4db6b15edb4481f9ccc8f08c45b179b/users/e1ff4cbdd194489a88a4fb4432312d5f/roles/84ee739146934c1ab32c501ff8093644 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 16:29:29.539092 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb6b2e3-86b0-4a90-80fc-034e2169a312 None None] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:29:29.539622 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb6b2e3-86b0-4a90-80fc-034e2169a312 None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:29:29.539731 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb6b2e3-86b0-4a90-80fc-034e2169a312 None None] PATH_INFO: `/` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:29:29.541031 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 603/1193] 10.222.0.57 () {54 vars in 926 bytes} [Fri Jul 1 16:29:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 16:29:29.546186 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9179c0a0-3865-478e-a136-bcf459efd548 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:29:29.546813 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9179c0a0-3865-478e-a136-bcf459efd548 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:29:29.546813 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9179c0a0-3865-478e-a136-bcf459efd548 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:29:29.563398 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-9179c0a0-3865-478e-a136-bcf459efd548 None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:29:29.564578 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-9179c0a0-3865-478e-a136-bcf459efd548 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:29:29.593409 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:29:29.630144 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 591/1194] 10.222.0.57 () {58 vars in 1023 bytes} [Fri Jul 1 16:29:29 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 01 16:29:29.653793 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb6b2e3-86b0-4a90-80fc-034e2169a312 None None] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:29:29.758013 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb6b2e3-86b0-4a90-80fc-034e2169a312 None None] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:29:29.791530 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7e2a4c-01c6-4988-b457-c6752dbc48c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:29:29.792244 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7e2a4c-01c6-4988-b457-c6752dbc48c1 None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:29:29.792529 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7e2a4c-01c6-4988-b457-c6752dbc48c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:29:29.792810 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7e2a4c-01c6-4988-b457-c6752dbc48c1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:29:29.802867 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7e2a4c-01c6-4988-b457-c6752dbc48c1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 01 16:29:29.810548 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application [None req-fe7e2a4c-01c6-4988-b457-c6752dbc48c1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 01 16:29:29.810548 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 16:29:29.810548 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 16:29:29.810548 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 16:29:29.810548 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 16:29:29.810548 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 16:29:29.810548 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 16:29:29.810548 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 16:29:29.810548 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 16:29:29.810548 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 16:29:29.810548 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 16:29:29.810548 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 16:29:29.810548 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 01 16:29:29.810548 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 01 16:29:29.810548 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 01 16:29:29.810548 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 01 16:29:29.812210 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 01 16:29:29.812210 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 01 16:29:29.812210 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 01 16:29:29.812210 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 01 16:29:29.812210 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 16:29:29.812210 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 16:29:29.812210 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 16:29:29.812210 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 16:29:29.812210 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 16:29:29.812210 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 16:29:29.812210 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 16:29:29.812210 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application with Lock( Jul 01 16:29:29.812210 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 16:29:29.812210 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self._enter() Jul 01 16:29:29.812210 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 16:29:29.812210 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 16:29:29.813116 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 16:29:29.813116 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.creator() Jul 01 16:29:29.813116 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 16:29:29.813116 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application created_value = creator( Jul 01 16:29:29.813116 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 01 16:29:29.813116 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 01 16:29:29.813116 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 01 16:29:29.813116 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 01 16:29:29.813116 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 01 16:29:29.813116 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 01 16:29:29.813116 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 01 16:29:29.813116 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application Jul 01 16:29:29.813116 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 604/1195] 10.222.0.57 () {56 vars in 1096 bytes} [Fri Jul 1 16:29:29 2022] GET /identity/v3/roles/admin => generated 83 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 16:29:29.818245 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9179c0a0-3865-478e-a136-bcf459efd548 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:29:29.915008 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9179c0a0-3865-478e-a136-bcf459efd548 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:29:29.948115 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b1b623-524c-416b-9084-bdecb1674b06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:29:29.948768 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b1b623-524c-416b-9084-bdecb1674b06 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:29:29.949050 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b1b623-524c-416b-9084-bdecb1674b06 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:29:29.949379 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b1b623-524c-416b-9084-bdecb1674b06 None admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:29:29.950195 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b1b623-524c-416b-9084-bdecb1674b06 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:29:29.950783 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b1b623-524c-416b-9084-bdecb1674b06 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:29:29.952135 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b1b623-524c-416b-9084-bdecb1674b06 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:29:29.960173 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 592/1196] 10.222.0.57 () {56 vars in 1099 bytes} [Fri Jul 1 16:29:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:29:29.965957 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7e2a4c-01c6-4988-b457-c6752dbc48c1 None admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:29:30.008815 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=100 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:29:30.064632 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7e2a4c-01c6-4988-b457-c6752dbc48c1 None admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:29:30.096949 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26361b1-9eeb-4b8c-a9e6-f5fa9f56d823 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:29:30.097812 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26361b1-9eeb-4b8c-a9e6-f5fa9f56d823 None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:29:30.098093 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26361b1-9eeb-4b8c-a9e6-f5fa9f56d823 None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:29:30.098390 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26361b1-9eeb-4b8c-a9e6-f5fa9f56d823 None admin] PATH_INFO: `/v3/users/e1ff4cbdd194489a88a4fb4432312d5f` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:29:30.099560 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f26361b1-9eeb-4b8c-a9e6-f5fa9f56d823 None admin] RBAC: Authorizing `identity:get_user(user_id=e1ff4cbdd194489a88a4fb4432312d5f)` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:29:30.100982 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f26361b1-9eeb-4b8c-a9e6-f5fa9f56d823 None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:29:30.102669 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 605/1197] 10.222.0.57 () {56 vars in 1177 bytes} [Fri Jul 1 16:29:29 2022] GET /identity/v3/users/e1ff4cbdd194489a88a4fb4432312d5f => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:29:30.108395 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b1b623-524c-416b-9084-bdecb1674b06 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:29:30.138987 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:29:30.206781 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b1b623-524c-416b-9084-bdecb1674b06 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:29:30.238988 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86037cde-f008-48b8-9fb6-bd4de38c6766 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:29:30.239622 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86037cde-f008-48b8-9fb6-bd4de38c6766 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:29:30.239906 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86037cde-f008-48b8-9fb6-bd4de38c6766 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:29:30.240182 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86037cde-f008-48b8-9fb6-bd4de38c6766 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:29:30.304638 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86037cde-f008-48b8-9fb6-bd4de38c6766 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:29:30.306076 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86037cde-f008-48b8-9fb6-bd4de38c6766 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:29:30.334302 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:29:30.406511 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 593/1198] 10.222.0.57 () {58 vars in 1282 bytes} [Fri Jul 1 16:29:30 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 01 16:29:30.412652 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26361b1-9eeb-4b8c-a9e6-f5fa9f56d823 None admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:29:30.511780 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26361b1-9eeb-4b8c-a9e6-f5fa9f56d823 None admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:29:30.522327 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=100 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:29:30.545226 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18af532-f3d4-4390-9483-3294cbf079b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:29:30.546609 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18af532-f3d4-4390-9483-3294cbf079b5 None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:29:30.546916 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18af532-f3d4-4390-9483-3294cbf079b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:29:30.547249 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18af532-f3d4-4390-9483-3294cbf079b5 None admin] PATH_INFO: `/v3/projects/f4db6b15edb4481f9ccc8f08c45b179b` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:29:30.548425 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18af532-f3d4-4390-9483-3294cbf079b5 None admin] RBAC: Authorizing `identity:get_project(project_id=f4db6b15edb4481f9ccc8f08c45b179b)` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:29:30.549850 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18af532-f3d4-4390-9483-3294cbf079b5 None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:29:30.551600 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 606/1199] 10.222.0.57 () {56 vars in 1186 bytes} [Fri Jul 1 16:29:30 2022] GET /identity/v3/projects/f4db6b15edb4481f9ccc8f08c45b179b => generated 361 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:29:30.557690 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86037cde-f008-48b8-9fb6-bd4de38c6766 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:29:30.653469 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86037cde-f008-48b8-9fb6-bd4de38c6766 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:29:30.681023 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:29:30.687822 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8dbe47-d506-46a5-8543-639f02902b4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:29:30.688458 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8dbe47-d506-46a5-8543-639f02902b4f None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:29:30.688739 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8dbe47-d506-46a5-8543-639f02902b4f None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:29:30.689015 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8dbe47-d506-46a5-8543-639f02902b4f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:29:30.689836 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f8dbe47-d506-46a5-8543-639f02902b4f None admin] RBAC: Adding query filter params (role.id=84ee739146934c1ab32c501ff8093644, scope.project.id=f4db6b15edb4481f9ccc8f08c45b179b, user.id=e1ff4cbdd194489a88a4fb4432312d5f) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:29:30.690146 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f8dbe47-d506-46a5-8543-639f02902b4f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:29:30.691601 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f8dbe47-d506-46a5-8543-639f02902b4f None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:29:30.697569 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 594/1200] 10.222.0.57 () {56 vars in 1374 bytes} [Fri Jul 1 16:29:30 2022] GET /identity/v3/role_assignments?user.id=e1ff4cbdd194489a88a4fb4432312d5f&scope.project.id=f4db6b15edb4481f9ccc8f08c45b179b&role.id=84ee739146934c1ab32c501ff8093644 => generated 608 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:29:40.541138 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=11 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:29:40.694966 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=3 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:30:27.817917 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c4925f-d976-4f5f-a658-855e226af5e9 None None] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:30:27.818529 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c4925f-d976-4f5f-a658-855e226af5e9 None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:30:27.818529 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c4925f-d976-4f5f-a658-855e226af5e9 None None] PATH_INFO: `/` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:30:27.820090 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 607/1201] 10.222.0.57 () {54 vars in 926 bytes} [Fri Jul 1 16:30:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 16:30:27.827467 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d166d23-bab7-4ebf-a884-15bbc083cb7a None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:30:27.827971 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d166d23-bab7-4ebf-a884-15bbc083cb7a None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:30:27.828077 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d166d23-bab7-4ebf-a884-15bbc083cb7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:30:27.848363 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-6d166d23-bab7-4ebf-a884-15bbc083cb7a None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:30:27.850009 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-6d166d23-bab7-4ebf-a884-15bbc083cb7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:30:27.932266 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 595/1202] 10.222.0.57 () {58 vars in 1023 bytes} [Fri Jul 1 16:30:27 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 01 16:30:27.966717 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c4925f-d976-4f5f-a658-855e226af5e9 None None] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:30:28.098977 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c4925f-d976-4f5f-a658-855e226af5e9 None None] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:30:28.150595 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf458eed-6580-4d2a-a845-cb707438dbf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:30:28.151412 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf458eed-6580-4d2a-a845-cb707438dbf5 None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:30:28.151711 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf458eed-6580-4d2a-a845-cb707438dbf5 None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:30:28.152510 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf458eed-6580-4d2a-a845-cb707438dbf5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:30:28.165579 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf458eed-6580-4d2a-a845-cb707438dbf5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:30:28.167162 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf458eed-6580-4d2a-a845-cb707438dbf5 None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:30:28.200286 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application [None req-bf458eed-6580-4d2a-a845-cb707438dbf5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 16:30:28.200286 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 16:30:28.200286 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 16:30:28.200286 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 16:30:28.200286 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 16:30:28.200286 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 16:30:28.200286 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 16:30:28.200286 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 16:30:28.200286 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 16:30:28.200286 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 16:30:28.200286 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 16:30:28.200286 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 16:30:28.200286 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 01 16:30:28.200286 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 01 16:30:28.200286 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 01 16:30:28.202151 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 01 16:30:28.202151 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 16:30:28.202151 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 16:30:28.202151 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 16:30:28.202151 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 16:30:28.202151 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 16:30:28.202151 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 16:30:28.202151 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 16:30:28.202151 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application with Lock( Jul 01 16:30:28.202151 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 16:30:28.202151 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self._enter() Jul 01 16:30:28.202151 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 16:30:28.202151 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 16:30:28.202151 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 16:30:28.202151 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.creator() Jul 01 16:30:28.202151 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 16:30:28.202151 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application created_value = creator( Jul 01 16:30:28.207731 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 01 16:30:28.207731 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 01 16:30:28.207731 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 16:30:28.207731 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application Jul 01 16:30:28.207731 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 608/1203] 10.222.0.57 () {56 vars in 1108 bytes} [Fri Jul 1 16:30:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 16:30:28.214417 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d166d23-bab7-4ebf-a884-15bbc083cb7a None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:30:28.348603 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d166d23-bab7-4ebf-a884-15bbc083cb7a None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:30:28.386331 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:30:28.403039 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b23b3d9-a2ce-4dcb-8b50-1280cacdb91a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:30:28.403815 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b23b3d9-a2ce-4dcb-8b50-1280cacdb91a None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:30:28.404105 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b23b3d9-a2ce-4dcb-8b50-1280cacdb91a None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:30:28.404374 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b23b3d9-a2ce-4dcb-8b50-1280cacdb91a None admin] PATH_INFO: `/v3/domains` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:30:28.405171 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b23b3d9-a2ce-4dcb-8b50-1280cacdb91a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:30:28.405509 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b23b3d9-a2ce-4dcb-8b50-1280cacdb91a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:30:28.406991 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b23b3d9-a2ce-4dcb-8b50-1280cacdb91a None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:30:28.430403 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 596/1204] 10.222.0.57 () {56 vars in 1109 bytes} [Fri Jul 1 16:30:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:30:28.432135 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf458eed-6580-4d2a-a845-cb707438dbf5 None admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:30:28.505304 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=100 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:30:28.575988 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf458eed-6580-4d2a-a845-cb707438dbf5 None admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:30:28.619800 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64118a23-fd92-4d41-bde7-e66ff717ece3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:30:28.621818 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64118a23-fd92-4d41-bde7-e66ff717ece3 None admin] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:30:28.622117 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64118a23-fd92-4d41-bde7-e66ff717ece3 None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:30:28.622472 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64118a23-fd92-4d41-bde7-e66ff717ece3 None admin] PATH_INFO: `/v3/users` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:30:28.623217 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64118a23-fd92-4d41-bde7-e66ff717ece3 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:30:28.624712 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64118a23-fd92-4d41-bde7-e66ff717ece3 None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:30:28.651683 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 609/1205] 10.222.0.57 () {60 vars in 1132 bytes} [Fri Jul 1 16:30:28 2022] POST /identity/v3/users => generated 251 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 16:30:30.973130 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3193f46f-c6d8-421d-9c0c-ff0ad45af623 None None] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:30:30.974770 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3193f46f-c6d8-421d-9c0c-ff0ad45af623 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:30:30.974770 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3193f46f-c6d8-421d-9c0c-ff0ad45af623 None None] PATH_INFO: `/` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:30:30.975587 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 597/1206] 10.222.0.57 () {54 vars in 926 bytes} [Fri Jul 1 16:30:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 16:30:30.981962 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa76cc66-8bfd-4c99-acf9-422db4a1ab9f None None] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:30:30.982458 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa76cc66-8bfd-4c99-acf9-422db4a1ab9f None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:30:30.982568 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa76cc66-8bfd-4c99-acf9-422db4a1ab9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:30:31.001696 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.auth.core [None req-fa76cc66-8bfd-4c99-acf9-422db4a1ab9f None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80312) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:30:31.003237 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-fa76cc66-8bfd-4c99-acf9-422db4a1ab9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:30:31.077747 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 610/1207] 10.222.0.57 () {58 vars in 1023 bytes} [Fri Jul 1 16:30:30 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 01 16:30:31.106019 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3193f46f-c6d8-421d-9c0c-ff0ad45af623 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:30:31.233406 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3193f46f-c6d8-421d-9c0c-ff0ad45af623 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:30:31.279104 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc037d0-a294-4459-b439-af0dad5db3a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:30:31.279920 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc037d0-a294-4459-b439-af0dad5db3a1 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:30:31.280208 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc037d0-a294-4459-b439-af0dad5db3a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:30:31.280504 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc037d0-a294-4459-b439-af0dad5db3a1 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:30:31.291985 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc037d0-a294-4459-b439-af0dad5db3a1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 01 16:30:31.299876 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application [None req-1cc037d0-a294-4459-b439-af0dad5db3a1 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 01 16:30:31.299876 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 16:30:31.299876 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 16:30:31.299876 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 16:30:31.299876 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 16:30:31.299876 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 16:30:31.299876 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 16:30:31.299876 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 16:30:31.299876 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 16:30:31.299876 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 16:30:31.299876 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 16:30:31.299876 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 16:30:31.299876 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 01 16:30:31.299876 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 01 16:30:31.299876 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 01 16:30:31.299876 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 01 16:30:31.307083 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 01 16:30:31.307083 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 01 16:30:31.307083 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 01 16:30:31.307083 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 01 16:30:31.307083 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 16:30:31.307083 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 16:30:31.307083 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 16:30:31.307083 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 16:30:31.307083 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 16:30:31.307083 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 16:30:31.307083 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 16:30:31.307083 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application with Lock( Jul 01 16:30:31.307083 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 16:30:31.307083 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._enter() Jul 01 16:30:31.307083 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 16:30:31.307083 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 16:30:31.308588 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 16:30:31.308588 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.creator() Jul 01 16:30:31.308588 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 16:30:31.308588 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application created_value = creator( Jul 01 16:30:31.308588 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 01 16:30:31.308588 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 01 16:30:31.308588 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 01 16:30:31.308588 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 01 16:30:31.308588 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 01 16:30:31.308588 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 01 16:30:31.308588 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 01 16:30:31.308588 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Jul 01 16:30:31.308588 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 598/1208] 10.222.0.57 () {56 vars in 1102 bytes} [Fri Jul 1 16:30:31 2022] GET /identity/v3/roles/service => generated 85 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 16:30:31.310201 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa76cc66-8bfd-4c99-acf9-422db4a1ab9f None None] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:30:31.360845 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:30:31.440737 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa76cc66-8bfd-4c99-acf9-422db4a1ab9f None None] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:30:31.486008 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1935579d-87cb-4885-ab46-d0f81aea4302 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:30:31.486861 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1935579d-87cb-4885-ab46-d0f81aea4302 None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:30:31.487277 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1935579d-87cb-4885-ab46-d0f81aea4302 None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:30:31.487669 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1935579d-87cb-4885-ab46-d0f81aea4302 None admin] PATH_INFO: `/v3/roles` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:30:31.488696 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1935579d-87cb-4885-ab46-d0f81aea4302 None admin] RBAC: Adding query filter params (name=service) {{(pid=80312) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:30:31.489144 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1935579d-87cb-4885-ab46-d0f81aea4302 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:30:31.490728 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1935579d-87cb-4885-ab46-d0f81aea4302 None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:30:31.502643 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 611/1209] 10.222.0.57 () {56 vars in 1103 bytes} [Fri Jul 1 16:30:31 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:30:31.508221 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc037d0-a294-4459-b439-af0dad5db3a1 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:30:31.581214 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:30:31.648195 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc037d0-a294-4459-b439-af0dad5db3a1 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:30:31.690644 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6b0e8d-9ead-4428-8e3e-f14822f2cdaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:30:31.691535 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6b0e8d-9ead-4428-8e3e-f14822f2cdaf None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:30:31.691822 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6b0e8d-9ead-4428-8e3e-f14822f2cdaf None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:30:31.692134 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6b0e8d-9ead-4428-8e3e-f14822f2cdaf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:30:31.704408 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f6b0e8d-9ead-4428-8e3e-f14822f2cdaf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:30:31.706059 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f6b0e8d-9ead-4428-8e3e-f14822f2cdaf None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:30:31.728633 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application [None req-7f6b0e8d-9ead-4428-8e3e-f14822f2cdaf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 16:30:31.728633 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 16:30:31.728633 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 16:30:31.728633 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 16:30:31.728633 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 16:30:31.728633 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 16:30:31.728633 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 16:30:31.728633 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 16:30:31.728633 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 16:30:31.728633 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 16:30:31.728633 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 16:30:31.728633 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 16:30:31.728633 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 01 16:30:31.728633 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 01 16:30:31.728633 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 01 16:30:31.732127 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 01 16:30:31.732127 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 16:30:31.732127 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 16:30:31.732127 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 16:30:31.732127 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 16:30:31.732127 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 16:30:31.732127 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 16:30:31.732127 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 16:30:31.732127 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application with Lock( Jul 01 16:30:31.732127 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 16:30:31.732127 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._enter() Jul 01 16:30:31.732127 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 16:30:31.732127 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 16:30:31.732127 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 16:30:31.732127 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.creator() Jul 01 16:30:31.732127 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 16:30:31.732127 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application created_value = creator( Jul 01 16:30:31.733120 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 01 16:30:31.733120 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 01 16:30:31.733120 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 16:30:31.733120 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Jul 01 16:30:31.737251 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 599/1210] 10.222.0.57 () {56 vars in 1108 bytes} [Fri Jul 1 16:30:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 16:30:31.737464 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1935579d-87cb-4885-ab46-d0f81aea4302 None admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:30:31.856870 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=100 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:30:31.859351 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1935579d-87cb-4885-ab46-d0f81aea4302 None admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:30:31.898950 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596f2af0-b6bd-45e7-9675-4626cc0d6655 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:30:31.899541 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596f2af0-b6bd-45e7-9675-4626cc0d6655 None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:30:31.899826 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596f2af0-b6bd-45e7-9675-4626cc0d6655 None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:30:31.900100 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596f2af0-b6bd-45e7-9675-4626cc0d6655 None admin] PATH_INFO: `/v3/domains` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:30:31.900825 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-596f2af0-b6bd-45e7-9675-4626cc0d6655 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80312) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:30:31.901135 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-596f2af0-b6bd-45e7-9675-4626cc0d6655 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:30:31.903167 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-596f2af0-b6bd-45e7-9675-4626cc0d6655 None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:30:31.918424 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 612/1211] 10.222.0.57 () {56 vars in 1109 bytes} [Fri Jul 1 16:30:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:30:31.923745 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6b0e8d-9ead-4428-8e3e-f14822f2cdaf None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:30:32.059154 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6b0e8d-9ead-4428-8e3e-f14822f2cdaf None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:30:32.061754 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:30:32.110395 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80de03bc-bd00-4123-9af8-fffc41e37fe4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:30:32.111214 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80de03bc-bd00-4123-9af8-fffc41e37fe4 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:30:32.111507 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80de03bc-bd00-4123-9af8-fffc41e37fe4 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:30:32.111792 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80de03bc-bd00-4123-9af8-fffc41e37fe4 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:30:32.122130 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80de03bc-bd00-4123-9af8-fffc41e37fe4 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:30:32.126458 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80de03bc-bd00-4123-9af8-fffc41e37fe4 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:30:32.145429 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application [None req-80de03bc-bd00-4123-9af8-fffc41e37fe4 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 01 16:30:32.145429 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 16:30:32.145429 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 16:30:32.145429 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 16:30:32.145429 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 16:30:32.145429 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 16:30:32.145429 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 16:30:32.145429 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 16:30:32.145429 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 16:30:32.145429 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 16:30:32.145429 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 16:30:32.145429 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 16:30:32.145429 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 01 16:30:32.145429 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 01 16:30:32.145429 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 01 16:30:32.145429 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 01 16:30:32.147853 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 16:30:32.147853 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 16:30:32.147853 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 01 16:30:32.147853 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 01 16:30:32.147853 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 01 16:30:32.147853 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 01 16:30:32.147853 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 16:30:32.147853 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 16:30:32.147853 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 16:30:32.147853 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 16:30:32.147853 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 16:30:32.147853 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application with Lock( Jul 01 16:30:32.147853 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 16:30:32.147853 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._enter() Jul 01 16:30:32.147853 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 16:30:32.147853 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 16:30:32.147853 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 16:30:32.147853 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.creator() Jul 01 16:30:32.150208 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 16:30:32.150208 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application created_value = creator( Jul 01 16:30:32.150208 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 01 16:30:32.150208 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 01 16:30:32.150208 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 01 16:30:32.150208 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 01 16:30:32.150208 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 01 16:30:32.150208 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 01 16:30:32.150208 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 01 16:30:32.150208 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Jul 01 16:30:32.150208 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 600/1212] 10.222.0.57 () {56 vars in 1108 bytes} [Fri Jul 1 16:30:31 2022] GET /identity/v3/users/placement => generated 87 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 16:30:32.156182 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596f2af0-b6bd-45e7-9675-4626cc0d6655 None admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:30:32.297441 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596f2af0-b6bd-45e7-9675-4626cc0d6655 None admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:30:32.336203 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d9cc0d-874a-4677-b077-cefb75d82393 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:30:32.336955 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d9cc0d-874a-4677-b077-cefb75d82393 None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:30:32.337323 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d9cc0d-874a-4677-b077-cefb75d82393 None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:30:32.337629 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d9cc0d-874a-4677-b077-cefb75d82393 None admin] PATH_INFO: `/v3/users` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:30:32.338628 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d9cc0d-874a-4677-b077-cefb75d82393 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80312) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:30:32.338947 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d9cc0d-874a-4677-b077-cefb75d82393 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:30:32.340644 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d9cc0d-874a-4677-b077-cefb75d82393 None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:30:32.360933 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 613/1213] 10.222.0.57 () {56 vars in 1143 bytes} [Fri Jul 1 16:30:32 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:30:32.368070 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80de03bc-bd00-4123-9af8-fffc41e37fe4 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:30:32.506215 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80de03bc-bd00-4123-9af8-fffc41e37fe4 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:30:32.550114 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0505f551-cbf2-45dc-a562-80a24d09a72e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:30:32.550964 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0505f551-cbf2-45dc-a562-80a24d09a72e None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:30:32.551259 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0505f551-cbf2-45dc-a562-80a24d09a72e None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:30:32.551589 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0505f551-cbf2-45dc-a562-80a24d09a72e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:30:32.594485 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:30:32.648380 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0505f551-cbf2-45dc-a562-80a24d09a72e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:30:32.649855 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0505f551-cbf2-45dc-a562-80a24d09a72e None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:30:32.767236 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 601/1214] 10.222.0.57 () {58 vars in 1282 bytes} [Fri Jul 1 16:30:32 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 399 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 01 16:30:32.773364 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d9cc0d-874a-4677-b077-cefb75d82393 None admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:30:32.795686 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=100 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:30:32.906383 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d9cc0d-874a-4677-b077-cefb75d82393 None admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:30:32.950475 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ccbc45-0717-4559-95c1-364d2d79facc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:30:32.951687 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ccbc45-0717-4559-95c1-364d2d79facc None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:30:32.951982 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ccbc45-0717-4559-95c1-364d2d79facc None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:30:32.952262 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ccbc45-0717-4559-95c1-364d2d79facc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:30:32.965124 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ccbc45-0717-4559-95c1-364d2d79facc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:30:32.966919 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ccbc45-0717-4559-95c1-364d2d79facc None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:30:32.987990 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application [None req-f6ccbc45-0717-4559-95c1-364d2d79facc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 16:30:32.987990 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 16:30:32.987990 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 16:30:32.987990 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 16:30:32.987990 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 16:30:32.987990 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 16:30:32.987990 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 16:30:32.987990 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 16:30:32.987990 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 16:30:32.987990 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 16:30:32.987990 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 16:30:32.987990 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 16:30:32.987990 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 01 16:30:32.987990 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 01 16:30:32.987990 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 01 16:30:32.989197 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 01 16:30:32.989197 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 16:30:32.989197 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 16:30:32.989197 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 16:30:32.989197 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 16:30:32.989197 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 16:30:32.989197 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 16:30:32.989197 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 16:30:32.989197 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application with Lock( Jul 01 16:30:32.989197 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 16:30:32.989197 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self._enter() Jul 01 16:30:32.989197 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 16:30:32.989197 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 16:30:32.989197 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 16:30:32.989197 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.creator() Jul 01 16:30:32.989197 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 16:30:32.989197 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application created_value = creator( Jul 01 16:30:32.990121 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 01 16:30:32.990121 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 01 16:30:32.990121 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 16:30:32.990121 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application Jul 01 16:30:32.990121 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 614/1215] 10.222.0.57 () {56 vars in 1108 bytes} [Fri Jul 1 16:30:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 01 16:30:32.996615 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0505f551-cbf2-45dc-a562-80a24d09a72e None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:30:33.066205 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:30:33.108568 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0505f551-cbf2-45dc-a562-80a24d09a72e None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:30:33.155292 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf12f4d-61aa-4fcf-81ad-fe2cb0becebd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:30:33.156128 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf12f4d-61aa-4fcf-81ad-fe2cb0becebd None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:30:33.156517 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf12f4d-61aa-4fcf-81ad-fe2cb0becebd None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:30:33.156797 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf12f4d-61aa-4fcf-81ad-fe2cb0becebd None admin] PATH_INFO: `/v3/domains` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:30:33.157532 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf12f4d-61aa-4fcf-81ad-fe2cb0becebd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:30:33.157838 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf12f4d-61aa-4fcf-81ad-fe2cb0becebd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:30:33.159253 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf12f4d-61aa-4fcf-81ad-fe2cb0becebd None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:30:33.171195 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 602/1216] 10.222.0.57 () {56 vars in 1109 bytes} [Fri Jul 1 16:30:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:30:33.177290 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ccbc45-0717-4559-95c1-364d2d79facc None admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:30:33.257868 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=100 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:30:33.298567 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ccbc45-0717-4559-95c1-364d2d79facc None admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:30:33.340880 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9febeb7f-63f2-492b-821a-5e29be4ded14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:30:33.341699 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9febeb7f-63f2-492b-821a-5e29be4ded14 None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:30:33.342047 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9febeb7f-63f2-492b-821a-5e29be4ded14 None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:30:33.342513 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9febeb7f-63f2-492b-821a-5e29be4ded14 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:30:33.349910 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9febeb7f-63f2-492b-821a-5e29be4ded14 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:30:33.351672 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9febeb7f-63f2-492b-821a-5e29be4ded14 None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:30:33.368058 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application [None req-9febeb7f-63f2-492b-821a-5e29be4ded14 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 01 16:30:33.368058 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 16:30:33.368058 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 16:30:33.368058 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 16:30:33.368058 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 16:30:33.368058 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 16:30:33.368058 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 16:30:33.368058 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 16:30:33.368058 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 16:30:33.368058 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 16:30:33.368058 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 16:30:33.368058 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 16:30:33.368058 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 01 16:30:33.368058 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 01 16:30:33.368058 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 01 16:30:33.369072 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 01 16:30:33.369072 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 16:30:33.369072 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 16:30:33.369072 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 16:30:33.369072 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 16:30:33.369072 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 16:30:33.369072 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 16:30:33.369072 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 16:30:33.369072 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application with Lock( Jul 01 16:30:33.369072 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 16:30:33.369072 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self._enter() Jul 01 16:30:33.369072 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 16:30:33.369072 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 16:30:33.369072 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 16:30:33.369072 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.creator() Jul 01 16:30:33.369072 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 16:30:33.369072 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application created_value = creator( Jul 01 16:30:33.370052 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 01 16:30:33.370052 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 01 16:30:33.370052 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 01 16:30:33.370052 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 01 16:30:33.370052 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 01 16:30:33.370052 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 01 16:30:33.370052 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 01 16:30:33.370052 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application Jul 01 16:30:33.370052 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 615/1217] 10.222.0.57 () {56 vars in 1111 bytes} [Fri Jul 1 16:30:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 16:30:33.375887 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf12f4d-61aa-4fcf-81ad-fe2cb0becebd None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:30:33.490507 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf12f4d-61aa-4fcf-81ad-fe2cb0becebd None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:30:33.516883 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:30:33.541563 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8e0b2c-f3bb-45ab-b14d-bdf5a20a4e89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:30:33.543266 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8e0b2c-f3bb-45ab-b14d-bdf5a20a4e89 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:30:33.543599 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8e0b2c-f3bb-45ab-b14d-bdf5a20a4e89 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:30:33.543924 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8e0b2c-f3bb-45ab-b14d-bdf5a20a4e89 None admin] PATH_INFO: `/v3/projects` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:30:33.544969 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f8e0b2c-f3bb-45ab-b14d-bdf5a20a4e89 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:30:33.545432 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f8e0b2c-f3bb-45ab-b14d-bdf5a20a4e89 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:30:33.547519 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f8e0b2c-f3bb-45ab-b14d-bdf5a20a4e89 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:30:33.563134 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 603/1218] 10.222.0.57 () {56 vars in 1148 bytes} [Fri Jul 1 16:30:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:30:33.569472 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9febeb7f-63f2-492b-821a-5e29be4ded14 None admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:30:33.695153 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9febeb7f-63f2-492b-821a-5e29be4ded14 None admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:30:33.730534 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=100 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:30:33.740970 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3cbf82f-3681-4c52-be93-77b3f8f49da1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:30:33.741643 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cbf82f-3681-4c52-be93-77b3f8f49da1 None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:30:33.741975 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cbf82f-3681-4c52-be93-77b3f8f49da1 None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:30:33.742292 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cbf82f-3681-4c52-be93-77b3f8f49da1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:30:33.743187 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3cbf82f-3681-4c52-be93-77b3f8f49da1 None admin] RBAC: Adding query filter params (role.id=e8809b67c9dc471daa1c2bd42c9a2baf, scope.project.id=e0788e6725a84e57a4174284c1845755, user.id=bbc38af18dec4e9b8143df23af6aad43) {{(pid=80312) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:30:33.743565 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3cbf82f-3681-4c52-be93-77b3f8f49da1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:30:33.745091 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3cbf82f-3681-4c52-be93-77b3f8f49da1 None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:30:33.762407 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 616/1219] 10.222.0.57 () {56 vars in 1374 bytes} [Fri Jul 1 16:30:33 2022] GET /identity/v3/role_assignments?user.id=bbc38af18dec4e9b8143df23af6aad43&scope.project.id=e0788e6725a84e57a4174284c1845755&role.id=e8809b67c9dc471daa1c2bd42c9a2baf => generated 259 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:30:35.972736 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06de520b-ebab-47ad-8a9d-e4187bf1a1bf None None] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:30:35.973338 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06de520b-ebab-47ad-8a9d-e4187bf1a1bf None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:30:35.973462 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06de520b-ebab-47ad-8a9d-e4187bf1a1bf None None] PATH_INFO: `/` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:30:35.975111 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 604/1220] 10.222.0.57 () {54 vars in 926 bytes} [Fri Jul 1 16:30:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 16:30:35.980866 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbfcb67-1224-4ab1-a391-a2e5ce15f38a None None] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:30:35.981788 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbfcb67-1224-4ab1-a391-a2e5ce15f38a None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:30:35.981788 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbfcb67-1224-4ab1-a391-a2e5ce15f38a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:30:35.999961 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.auth.core [None req-fdbfcb67-1224-4ab1-a391-a2e5ce15f38a None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80312) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:30:36.001377 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-fdbfcb67-1224-4ab1-a391-a2e5ce15f38a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:30:36.082454 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 617/1221] 10.222.0.57 () {58 vars in 1023 bytes} [Fri Jul 1 16:30:35 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 01 16:30:36.107028 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06de520b-ebab-47ad-8a9d-e4187bf1a1bf None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:30:36.221228 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06de520b-ebab-47ad-8a9d-e4187bf1a1bf None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:30:36.260321 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699d3982-e242-4920-8e8b-8aebd990514a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:30:36.261836 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699d3982-e242-4920-8e8b-8aebd990514a None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:30:36.262253 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699d3982-e242-4920-8e8b-8aebd990514a None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:30:36.262668 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699d3982-e242-4920-8e8b-8aebd990514a None admin] PATH_INFO: `/v3/roles/service` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:30:36.274664 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-699d3982-e242-4920-8e8b-8aebd990514a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 01 16:30:36.283890 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application [None req-699d3982-e242-4920-8e8b-8aebd990514a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 01 16:30:36.283890 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 16:30:36.283890 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 16:30:36.283890 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 16:30:36.283890 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 16:30:36.283890 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 16:30:36.283890 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 16:30:36.283890 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 16:30:36.283890 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 16:30:36.283890 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 16:30:36.283890 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 16:30:36.283890 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 16:30:36.283890 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 01 16:30:36.283890 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 01 16:30:36.283890 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 01 16:30:36.283890 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 01 16:30:36.285029 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 01 16:30:36.285029 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 01 16:30:36.285029 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 01 16:30:36.285029 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 01 16:30:36.285029 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 16:30:36.285029 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 16:30:36.285029 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 16:30:36.285029 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 16:30:36.285029 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 16:30:36.285029 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 16:30:36.285029 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 16:30:36.285029 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application with Lock( Jul 01 16:30:36.285029 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 16:30:36.285029 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._enter() Jul 01 16:30:36.285029 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 16:30:36.285029 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 16:30:36.291274 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 16:30:36.291274 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.creator() Jul 01 16:30:36.291274 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 16:30:36.291274 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application created_value = creator( Jul 01 16:30:36.291274 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 01 16:30:36.291274 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 01 16:30:36.291274 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 01 16:30:36.291274 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 01 16:30:36.291274 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 01 16:30:36.291274 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 01 16:30:36.291274 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 01 16:30:36.291274 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Jul 01 16:30:36.291274 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 605/1222] 10.222.0.57 () {56 vars in 1102 bytes} [Fri Jul 1 16:30:36 2022] GET /identity/v3/roles/service => generated 85 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 16:30:36.292551 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdbfcb67-1224-4ab1-a391-a2e5ce15f38a None None] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:30:36.405107 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdbfcb67-1224-4ab1-a391-a2e5ce15f38a None None] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:30:36.428790 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=100 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:30:36.441497 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f96a9ef-0bac-495b-9461-e1acc23e1c23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:30:36.442173 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f96a9ef-0bac-495b-9461-e1acc23e1c23 None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:30:36.442505 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f96a9ef-0bac-495b-9461-e1acc23e1c23 None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:30:36.442784 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f96a9ef-0bac-495b-9461-e1acc23e1c23 None admin] PATH_INFO: `/v3/roles` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:30:36.443548 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f96a9ef-0bac-495b-9461-e1acc23e1c23 None admin] RBAC: Adding query filter params (name=service) {{(pid=80312) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:30:36.443864 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f96a9ef-0bac-495b-9461-e1acc23e1c23 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:30:36.445279 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f96a9ef-0bac-495b-9461-e1acc23e1c23 None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:30:36.454009 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 618/1223] 10.222.0.57 () {56 vars in 1103 bytes} [Fri Jul 1 16:30:36 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:30:36.460209 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699d3982-e242-4920-8e8b-8aebd990514a None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:30:36.509031 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:30:36.578819 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699d3982-e242-4920-8e8b-8aebd990514a None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:30:36.627307 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11003de8-e59d-4850-87bb-ea13b8ebe3a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:30:36.628462 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11003de8-e59d-4850-87bb-ea13b8ebe3a2 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:30:36.628917 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11003de8-e59d-4850-87bb-ea13b8ebe3a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:30:36.629391 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11003de8-e59d-4850-87bb-ea13b8ebe3a2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:30:36.647891 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11003de8-e59d-4850-87bb-ea13b8ebe3a2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:30:36.650565 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11003de8-e59d-4850-87bb-ea13b8ebe3a2 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:30:36.682948 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application [None req-11003de8-e59d-4850-87bb-ea13b8ebe3a2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 16:30:36.682948 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 16:30:36.682948 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 16:30:36.682948 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 16:30:36.682948 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 16:30:36.682948 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 16:30:36.682948 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 16:30:36.682948 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 16:30:36.682948 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 16:30:36.682948 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 16:30:36.682948 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 16:30:36.682948 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 16:30:36.682948 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 01 16:30:36.682948 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 01 16:30:36.682948 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 01 16:30:36.683909 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 01 16:30:36.683909 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 16:30:36.683909 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 16:30:36.683909 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 16:30:36.683909 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 16:30:36.683909 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 16:30:36.683909 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 16:30:36.683909 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 16:30:36.683909 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application with Lock( Jul 01 16:30:36.683909 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 16:30:36.683909 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._enter() Jul 01 16:30:36.683909 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 16:30:36.683909 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 16:30:36.683909 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 16:30:36.683909 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.creator() Jul 01 16:30:36.683909 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 16:30:36.683909 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application created_value = creator( Jul 01 16:30:36.684868 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 01 16:30:36.684868 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 01 16:30:36.684868 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 16:30:36.684868 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Jul 01 16:30:36.685107 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 606/1224] 10.222.0.57 () {56 vars in 1108 bytes} [Fri Jul 1 16:30:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 16:30:36.692617 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f96a9ef-0bac-495b-9461-e1acc23e1c23 None admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:30:36.814637 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f96a9ef-0bac-495b-9461-e1acc23e1c23 None admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:30:36.855350 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1efa1f0c-55c3-47e4-b2ca-90f1b5137686 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:30:36.856030 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efa1f0c-55c3-47e4-b2ca-90f1b5137686 None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:30:36.856317 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efa1f0c-55c3-47e4-b2ca-90f1b5137686 None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:30:36.856597 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efa1f0c-55c3-47e4-b2ca-90f1b5137686 None admin] PATH_INFO: `/v3/domains` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:30:36.857332 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1efa1f0c-55c3-47e4-b2ca-90f1b5137686 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80312) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:30:36.857650 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1efa1f0c-55c3-47e4-b2ca-90f1b5137686 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:30:36.859088 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1efa1f0c-55c3-47e4-b2ca-90f1b5137686 None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:30:36.873297 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 619/1225] 10.222.0.57 () {56 vars in 1109 bytes} [Fri Jul 1 16:30:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:30:36.879840 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11003de8-e59d-4850-87bb-ea13b8ebe3a2 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:30:36.992067 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:30:37.000502 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11003de8-e59d-4850-87bb-ea13b8ebe3a2 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:30:37.042741 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce8fd91-34cb-4086-9a7a-92dd0ac02a5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:30:37.043651 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce8fd91-34cb-4086-9a7a-92dd0ac02a5d None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:30:37.043973 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce8fd91-34cb-4086-9a7a-92dd0ac02a5d None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:30:37.044324 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce8fd91-34cb-4086-9a7a-92dd0ac02a5d None admin] PATH_INFO: `/v3/users/placement` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:30:37.056489 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce8fd91-34cb-4086-9a7a-92dd0ac02a5d None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:30:37.060189 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce8fd91-34cb-4086-9a7a-92dd0ac02a5d None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:30:37.090575 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application [None req-0ce8fd91-34cb-4086-9a7a-92dd0ac02a5d None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 01 16:30:37.090575 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 16:30:37.090575 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 16:30:37.090575 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 16:30:37.090575 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 16:30:37.090575 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 16:30:37.090575 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 16:30:37.090575 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 16:30:37.090575 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 16:30:37.090575 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 16:30:37.090575 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 16:30:37.090575 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 16:30:37.090575 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 01 16:30:37.090575 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 01 16:30:37.090575 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 01 16:30:37.090575 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 01 16:30:37.093437 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 16:30:37.093437 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 16:30:37.093437 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 01 16:30:37.093437 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 01 16:30:37.093437 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 01 16:30:37.093437 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 01 16:30:37.093437 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 16:30:37.093437 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 16:30:37.093437 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 16:30:37.093437 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 16:30:37.093437 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 16:30:37.093437 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application with Lock( Jul 01 16:30:37.093437 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 16:30:37.093437 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._enter() Jul 01 16:30:37.093437 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 16:30:37.093437 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 16:30:37.093437 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 16:30:37.093437 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.creator() Jul 01 16:30:37.094423 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 16:30:37.094423 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application created_value = creator( Jul 01 16:30:37.094423 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 01 16:30:37.094423 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 01 16:30:37.094423 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 01 16:30:37.094423 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 01 16:30:37.094423 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 01 16:30:37.094423 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 01 16:30:37.094423 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 01 16:30:37.094423 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Jul 01 16:30:37.096773 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 607/1226] 10.222.0.57 () {56 vars in 1108 bytes} [Fri Jul 1 16:30:36 2022] GET /identity/v3/users/placement => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 16:30:37.105013 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1efa1f0c-55c3-47e4-b2ca-90f1b5137686 None admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:30:37.172653 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=100 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:30:37.233105 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1efa1f0c-55c3-47e4-b2ca-90f1b5137686 None admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:30:37.278837 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c1df96-c7ed-4c55-a578-a378ef677f8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:30:37.279661 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c1df96-c7ed-4c55-a578-a378ef677f8a None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:30:37.280145 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c1df96-c7ed-4c55-a578-a378ef677f8a None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:30:37.280266 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c1df96-c7ed-4c55-a578-a378ef677f8a None admin] PATH_INFO: `/v3/users` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:30:37.281120 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c1df96-c7ed-4c55-a578-a378ef677f8a None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80312) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:30:37.281377 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c1df96-c7ed-4c55-a578-a378ef677f8a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:30:37.282886 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c1df96-c7ed-4c55-a578-a378ef677f8a None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:30:37.304820 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 620/1227] 10.222.0.57 () {56 vars in 1143 bytes} [Fri Jul 1 16:30:37 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:30:37.311915 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce8fd91-34cb-4086-9a7a-92dd0ac02a5d None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:30:37.440815 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce8fd91-34cb-4086-9a7a-92dd0ac02a5d None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:30:37.494459 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0373bbd3-2234-4f9e-bc0c-7bf5152529c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:30:37.496674 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0373bbd3-2234-4f9e-bc0c-7bf5152529c1 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:30:37.497123 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0373bbd3-2234-4f9e-bc0c-7bf5152529c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:30:37.497660 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0373bbd3-2234-4f9e-bc0c-7bf5152529c1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:30:37.514146 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0373bbd3-2234-4f9e-bc0c-7bf5152529c1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:30:37.516450 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0373bbd3-2234-4f9e-bc0c-7bf5152529c1 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:30:37.546370 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application [None req-0373bbd3-2234-4f9e-bc0c-7bf5152529c1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 16:30:37.546370 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 16:30:37.546370 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 16:30:37.546370 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 16:30:37.546370 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 16:30:37.546370 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 16:30:37.546370 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 16:30:37.546370 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 16:30:37.546370 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 16:30:37.546370 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 16:30:37.546370 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 16:30:37.546370 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 16:30:37.546370 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 01 16:30:37.546370 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 01 16:30:37.546370 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 01 16:30:37.547385 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 01 16:30:37.547385 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 16:30:37.547385 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 16:30:37.547385 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 16:30:37.547385 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 16:30:37.547385 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 16:30:37.547385 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 16:30:37.547385 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 16:30:37.547385 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application with Lock( Jul 01 16:30:37.547385 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 16:30:37.547385 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._enter() Jul 01 16:30:37.547385 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 16:30:37.547385 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 16:30:37.547385 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 16:30:37.547385 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.creator() Jul 01 16:30:37.547385 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 16:30:37.547385 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application created_value = creator( Jul 01 16:30:37.548312 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 01 16:30:37.548312 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 01 16:30:37.548312 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 16:30:37.548312 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Jul 01 16:30:37.548312 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 608/1228] 10.222.0.57 () {56 vars in 1108 bytes} [Fri Jul 1 16:30:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 16:30:37.554934 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c1df96-c7ed-4c55-a578-a378ef677f8a None admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:30:37.675149 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c1df96-c7ed-4c55-a578-a378ef677f8a None admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:30:37.677551 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=100 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:30:37.716181 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54163be-fe97-42ba-90b7-225f03b37989 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:30:37.716997 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54163be-fe97-42ba-90b7-225f03b37989 None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:30:37.717338 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54163be-fe97-42ba-90b7-225f03b37989 None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:30:37.717551 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54163be-fe97-42ba-90b7-225f03b37989 None admin] PATH_INFO: `/v3/domains` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:30:37.718370 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c54163be-fe97-42ba-90b7-225f03b37989 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80312) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:30:37.718805 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c54163be-fe97-42ba-90b7-225f03b37989 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:30:37.720413 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c54163be-fe97-42ba-90b7-225f03b37989 None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:30:37.734088 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 621/1229] 10.222.0.57 () {56 vars in 1109 bytes} [Fri Jul 1 16:30:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:30:37.740497 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0373bbd3-2234-4f9e-bc0c-7bf5152529c1 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:30:37.752826 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:30:37.873563 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0373bbd3-2234-4f9e-bc0c-7bf5152529c1 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:30:37.926625 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f6ff83-d74a-4d16-bbcc-85eaa0dc00b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:30:37.927549 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f6ff83-d74a-4d16-bbcc-85eaa0dc00b8 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:30:37.927925 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f6ff83-d74a-4d16-bbcc-85eaa0dc00b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:30:37.928237 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f6ff83-d74a-4d16-bbcc-85eaa0dc00b8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:30:37.936348 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f6ff83-d74a-4d16-bbcc-85eaa0dc00b8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:30:37.937808 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f6ff83-d74a-4d16-bbcc-85eaa0dc00b8 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:30:37.953137 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application [None req-48f6ff83-d74a-4d16-bbcc-85eaa0dc00b8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 01 16:30:37.953137 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 16:30:37.953137 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 16:30:37.953137 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 16:30:37.953137 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 16:30:37.953137 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 16:30:37.953137 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 16:30:37.953137 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 16:30:37.953137 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 16:30:37.953137 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 16:30:37.953137 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 16:30:37.953137 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 16:30:37.953137 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 01 16:30:37.953137 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 01 16:30:37.953137 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 01 16:30:37.954209 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 01 16:30:37.954209 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 16:30:37.954209 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 16:30:37.954209 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 16:30:37.954209 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 16:30:37.954209 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 16:30:37.954209 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 16:30:37.954209 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 16:30:37.954209 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application with Lock( Jul 01 16:30:37.954209 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 16:30:37.954209 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._enter() Jul 01 16:30:37.954209 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 16:30:37.954209 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 16:30:37.954209 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 16:30:37.954209 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.creator() Jul 01 16:30:37.954209 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 16:30:37.954209 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application created_value = creator( Jul 01 16:30:37.955190 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 01 16:30:37.955190 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 01 16:30:37.955190 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 01 16:30:37.955190 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 01 16:30:37.955190 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 01 16:30:37.955190 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 01 16:30:37.955190 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 01 16:30:37.955190 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Jul 01 16:30:37.955190 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 609/1230] 10.222.0.57 () {56 vars in 1111 bytes} [Fri Jul 1 16:30:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 16:30:37.960916 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54163be-fe97-42ba-90b7-225f03b37989 None admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:30:38.072541 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54163be-fe97-42ba-90b7-225f03b37989 None admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:30:38.111426 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4083fb84-b71e-41a9-80f3-ea02405f7335 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:30:38.112178 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4083fb84-b71e-41a9-80f3-ea02405f7335 None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:30:38.112297 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4083fb84-b71e-41a9-80f3-ea02405f7335 None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:30:38.112573 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4083fb84-b71e-41a9-80f3-ea02405f7335 None admin] PATH_INFO: `/v3/projects` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:30:38.113377 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4083fb84-b71e-41a9-80f3-ea02405f7335 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80312) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:30:38.113696 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4083fb84-b71e-41a9-80f3-ea02405f7335 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:30:38.115174 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4083fb84-b71e-41a9-80f3-ea02405f7335 None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:30:38.127826 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 622/1231] 10.222.0.57 () {56 vars in 1148 bytes} [Fri Jul 1 16:30:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:30:38.134024 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f6ff83-d74a-4d16-bbcc-85eaa0dc00b8 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:30:38.214726 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:30:38.254390 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f6ff83-d74a-4d16-bbcc-85eaa0dc00b8 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:30:38.298587 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fad16a2-b530-49fb-b925-cf0dcd6721a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:30:38.299586 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fad16a2-b530-49fb-b925-cf0dcd6721a3 None admin] REQUEST_METHOD: `PUT` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:30:38.300008 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fad16a2-b530-49fb-b925-cf0dcd6721a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:30:38.300301 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fad16a2-b530-49fb-b925-cf0dcd6721a3 None admin] PATH_INFO: `/v3/projects/e0788e6725a84e57a4174284c1845755/users/bbc38af18dec4e9b8143df23af6aad43/roles/e8809b67c9dc471daa1c2bd42c9a2baf` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:30:38.319602 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fad16a2-b530-49fb-b925-cf0dcd6721a3 None admin] RBAC: Authorizing `identity:create_grant(project_id=e0788e6725a84e57a4174284c1845755, user_id=bbc38af18dec4e9b8143df23af6aad43, role_id=e8809b67c9dc471daa1c2bd42c9a2baf)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:30:38.321144 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fad16a2-b530-49fb-b925-cf0dcd6721a3 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:30:38.341026 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 610/1232] 10.222.0.57 () {58 vars in 1439 bytes} [Fri Jul 1 16:30:38 2022] PUT /identity/v3/projects/e0788e6725a84e57a4174284c1845755/users/bbc38af18dec4e9b8143df23af6aad43/roles/e8809b67c9dc471daa1c2bd42c9a2baf => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 16:30:40.616496 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8ace86-3d0c-4f8e-bd1d-ff1fe5bcd24d None None] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:30:40.617139 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8ace86-3d0c-4f8e-bd1d-ff1fe5bcd24d None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:30:40.617139 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8ace86-3d0c-4f8e-bd1d-ff1fe5bcd24d None None] PATH_INFO: `/` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:30:40.618630 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 623/1233] 10.222.0.57 () {54 vars in 926 bytes} [Fri Jul 1 16:30:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 16:30:40.624460 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fa8955-c8bc-4705-a2fe-154c100778ed None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:30:40.624910 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fa8955-c8bc-4705-a2fe-154c100778ed None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:30:40.625023 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fa8955-c8bc-4705-a2fe-154c100778ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:30:40.649082 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-27fa8955-c8bc-4705-a2fe-154c100778ed None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:30:40.650624 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-27fa8955-c8bc-4705-a2fe-154c100778ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:30:40.742485 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 611/1234] 10.222.0.57 () {58 vars in 1023 bytes} [Fri Jul 1 16:30:40 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 01 16:30:40.767408 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8ace86-3d0c-4f8e-bd1d-ff1fe5bcd24d None None] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:30:40.801313 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=100 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:30:40.904281 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8ace86-3d0c-4f8e-bd1d-ff1fe5bcd24d None None] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:30:40.964606 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ff8f92-ade1-45de-b2e3-0a424ed43fca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:30:40.965663 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ff8f92-ade1-45de-b2e3-0a424ed43fca None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:30:40.966024 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ff8f92-ade1-45de-b2e3-0a424ed43fca None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:30:40.966415 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ff8f92-ade1-45de-b2e3-0a424ed43fca None admin] PATH_INFO: `/v3/roles/service` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:30:40.980774 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ff8f92-ade1-45de-b2e3-0a424ed43fca None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 01 16:30:40.989525 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application [None req-c6ff8f92-ade1-45de-b2e3-0a424ed43fca None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 01 16:30:40.989525 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 16:30:40.989525 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 16:30:40.989525 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 16:30:40.989525 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 16:30:40.989525 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 16:30:40.989525 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 16:30:40.989525 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 16:30:40.989525 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 16:30:40.989525 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 16:30:40.989525 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 16:30:40.989525 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 16:30:40.989525 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 01 16:30:40.989525 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 01 16:30:40.989525 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 01 16:30:40.989525 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 01 16:30:40.990785 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 01 16:30:40.990785 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 01 16:30:40.990785 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 01 16:30:40.990785 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 01 16:30:40.990785 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 16:30:40.990785 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 16:30:40.990785 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 16:30:40.990785 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 16:30:40.990785 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 16:30:40.990785 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 16:30:40.990785 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 16:30:40.990785 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application with Lock( Jul 01 16:30:40.990785 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 16:30:40.990785 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self._enter() Jul 01 16:30:40.990785 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 16:30:40.990785 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 16:30:40.995608 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 16:30:40.995608 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.creator() Jul 01 16:30:40.995608 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 16:30:40.995608 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application created_value = creator( Jul 01 16:30:40.995608 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 01 16:30:40.995608 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 01 16:30:40.995608 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 01 16:30:40.995608 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 01 16:30:40.995608 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 01 16:30:40.995608 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 01 16:30:40.995608 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 01 16:30:40.995608 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application Jul 01 16:30:40.995608 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 624/1235] 10.222.0.57 () {56 vars in 1102 bytes} [Fri Jul 1 16:30:40 2022] GET /identity/v3/roles/service => generated 85 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 16:30:40.997116 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27fa8955-c8bc-4705-a2fe-154c100778ed None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:30:41.076063 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:30:41.149392 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27fa8955-c8bc-4705-a2fe-154c100778ed None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:30:41.197620 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3366e2-4d85-49ec-bd2c-af8455cd4dad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:30:41.198928 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3366e2-4d85-49ec-bd2c-af8455cd4dad None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:30:41.199361 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3366e2-4d85-49ec-bd2c-af8455cd4dad None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:30:41.199902 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3366e2-4d85-49ec-bd2c-af8455cd4dad None admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:30:41.200792 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb3366e2-4d85-49ec-bd2c-af8455cd4dad None admin] RBAC: Adding query filter params (name=service) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:30:41.201107 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb3366e2-4d85-49ec-bd2c-af8455cd4dad None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:30:41.202537 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb3366e2-4d85-49ec-bd2c-af8455cd4dad None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:30:41.214627 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 612/1236] 10.222.0.57 () {56 vars in 1103 bytes} [Fri Jul 1 16:30:40 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:30:41.221020 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ff8f92-ade1-45de-b2e3-0a424ed43fca None admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:30:41.334767 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=100 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:30:41.363774 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ff8f92-ade1-45de-b2e3-0a424ed43fca None admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:30:41.407212 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6008f981-f328-40e9-8859-6cad64725423 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:30:41.408215 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6008f981-f328-40e9-8859-6cad64725423 None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:30:41.408549 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6008f981-f328-40e9-8859-6cad64725423 None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:30:41.408877 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6008f981-f328-40e9-8859-6cad64725423 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:30:41.422924 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6008f981-f328-40e9-8859-6cad64725423 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:30:41.425352 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6008f981-f328-40e9-8859-6cad64725423 None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:30:41.446248 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application [None req-6008f981-f328-40e9-8859-6cad64725423 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 16:30:41.446248 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 16:30:41.446248 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 16:30:41.446248 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 16:30:41.446248 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 16:30:41.446248 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 16:30:41.446248 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 16:30:41.446248 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 16:30:41.446248 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 16:30:41.446248 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 16:30:41.446248 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 16:30:41.446248 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 16:30:41.446248 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 01 16:30:41.446248 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 01 16:30:41.446248 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 01 16:30:41.447601 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 01 16:30:41.447601 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 16:30:41.447601 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 16:30:41.447601 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 16:30:41.447601 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 16:30:41.447601 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 16:30:41.447601 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 16:30:41.447601 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 16:30:41.447601 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application with Lock( Jul 01 16:30:41.447601 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 16:30:41.447601 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self._enter() Jul 01 16:30:41.447601 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 16:30:41.447601 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 16:30:41.447601 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 16:30:41.447601 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.creator() Jul 01 16:30:41.447601 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 16:30:41.447601 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application created_value = creator( Jul 01 16:30:41.449819 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 01 16:30:41.449819 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 01 16:30:41.449819 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 16:30:41.449819 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application Jul 01 16:30:41.449819 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 625/1237] 10.222.0.57 () {56 vars in 1108 bytes} [Fri Jul 1 16:30:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 16:30:41.454007 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3366e2-4d85-49ec-bd2c-af8455cd4dad None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:30:41.607502 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3366e2-4d85-49ec-bd2c-af8455cd4dad None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:30:41.621173 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:30:41.658780 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e86848-dc97-4e06-afa2-e9815aead65b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:30:41.661007 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e86848-dc97-4e06-afa2-e9815aead65b None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:30:41.661819 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e86848-dc97-4e06-afa2-e9815aead65b None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:30:41.662632 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e86848-dc97-4e06-afa2-e9815aead65b None admin] PATH_INFO: `/v3/domains` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:30:41.664336 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e86848-dc97-4e06-afa2-e9815aead65b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:30:41.665187 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e86848-dc97-4e06-afa2-e9815aead65b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:30:41.668213 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e86848-dc97-4e06-afa2-e9815aead65b None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:30:41.694043 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 613/1238] 10.222.0.57 () {56 vars in 1109 bytes} [Fri Jul 1 16:30:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:30:41.700432 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6008f981-f328-40e9-8859-6cad64725423 None admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:30:41.885408 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6008f981-f328-40e9-8859-6cad64725423 None admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:30:41.933648 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc75301b-05ad-4085-b2bd-05bb9b980efd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:30:41.934634 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc75301b-05ad-4085-b2bd-05bb9b980efd None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:30:41.934996 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc75301b-05ad-4085-b2bd-05bb9b980efd None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:30:41.935311 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc75301b-05ad-4085-b2bd-05bb9b980efd None admin] PATH_INFO: `/v3/users/placement` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:30:41.939874 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=100 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:30:41.948614 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc75301b-05ad-4085-b2bd-05bb9b980efd None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:30:41.950415 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc75301b-05ad-4085-b2bd-05bb9b980efd None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:30:41.973471 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application [None req-fc75301b-05ad-4085-b2bd-05bb9b980efd None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 01 16:30:41.973471 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 16:30:41.973471 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 16:30:41.973471 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 16:30:41.973471 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 16:30:41.973471 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 16:30:41.973471 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 16:30:41.973471 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 16:30:41.973471 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 16:30:41.973471 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 16:30:41.973471 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 16:30:41.973471 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 16:30:41.973471 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 01 16:30:41.973471 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 01 16:30:41.973471 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 01 16:30:41.973471 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 01 16:30:41.974700 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 16:30:41.974700 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 16:30:41.974700 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 01 16:30:41.974700 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 01 16:30:41.974700 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 01 16:30:41.974700 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 01 16:30:41.974700 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 16:30:41.974700 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 16:30:41.974700 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 16:30:41.974700 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 16:30:41.974700 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 16:30:41.974700 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application with Lock( Jul 01 16:30:41.974700 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 16:30:41.974700 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self._enter() Jul 01 16:30:41.974700 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 16:30:41.974700 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 16:30:41.974700 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 16:30:41.974700 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.creator() Jul 01 16:30:41.978168 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 16:30:41.978168 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application created_value = creator( Jul 01 16:30:41.978168 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 01 16:30:41.978168 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 01 16:30:41.978168 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 01 16:30:41.978168 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 01 16:30:41.978168 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 01 16:30:41.978168 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 01 16:30:41.978168 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 01 16:30:41.978168 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application Jul 01 16:30:41.978168 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 626/1239] 10.222.0.57 () {56 vars in 1108 bytes} [Fri Jul 1 16:30:41 2022] GET /identity/v3/users/placement => generated 87 bytes in 275 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 16:30:41.982285 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e86848-dc97-4e06-afa2-e9815aead65b None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:30:42.137820 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e86848-dc97-4e06-afa2-e9815aead65b None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:30:42.186442 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5456e8c-0599-49ff-a7b2-c145e87f9861 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:30:42.187270 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5456e8c-0599-49ff-a7b2-c145e87f9861 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:30:42.187572 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5456e8c-0599-49ff-a7b2-c145e87f9861 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:30:42.187890 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5456e8c-0599-49ff-a7b2-c145e87f9861 None admin] PATH_INFO: `/v3/users` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:30:42.188752 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5456e8c-0599-49ff-a7b2-c145e87f9861 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:30:42.189073 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5456e8c-0599-49ff-a7b2-c145e87f9861 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:30:42.190649 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5456e8c-0599-49ff-a7b2-c145e87f9861 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:30:42.217210 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 614/1240] 10.222.0.57 () {56 vars in 1143 bytes} [Fri Jul 1 16:30:41 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:30:42.225127 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc75301b-05ad-4085-b2bd-05bb9b980efd None admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:30:42.389596 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc75301b-05ad-4085-b2bd-05bb9b980efd None admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:30:42.433650 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6cedb8-9959-4159-b392-3d3c5c1edf88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:30:42.434314 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6cedb8-9959-4159-b392-3d3c5c1edf88 None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:30:42.434686 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6cedb8-9959-4159-b392-3d3c5c1edf88 None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:30:42.434964 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6cedb8-9959-4159-b392-3d3c5c1edf88 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:30:42.532796 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=100 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:30:42.544348 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b6cedb8-9959-4159-b392-3d3c5c1edf88 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:30:42.547860 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b6cedb8-9959-4159-b392-3d3c5c1edf88 None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:30:42.688189 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 627/1241] 10.222.0.57 () {58 vars in 1282 bytes} [Fri Jul 1 16:30:42 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 464 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 01 16:30:42.694459 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5456e8c-0599-49ff-a7b2-c145e87f9861 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:30:42.728645 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:30:42.828327 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5456e8c-0599-49ff-a7b2-c145e87f9861 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:30:42.878220 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44dde3ec-b616-4b34-b719-1c0237200bdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:30:42.879200 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44dde3ec-b616-4b34-b719-1c0237200bdd None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:30:42.879700 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44dde3ec-b616-4b34-b719-1c0237200bdd None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:30:42.880201 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44dde3ec-b616-4b34-b719-1c0237200bdd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:30:42.896364 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44dde3ec-b616-4b34-b719-1c0237200bdd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:30:42.898509 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44dde3ec-b616-4b34-b719-1c0237200bdd None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:30:42.923342 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application [None req-44dde3ec-b616-4b34-b719-1c0237200bdd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 16:30:42.923342 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 16:30:42.923342 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 16:30:42.923342 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 16:30:42.923342 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 16:30:42.923342 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 16:30:42.923342 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 16:30:42.923342 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 16:30:42.923342 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 16:30:42.923342 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 16:30:42.923342 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 16:30:42.923342 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 16:30:42.923342 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 01 16:30:42.923342 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 01 16:30:42.923342 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 01 16:30:42.924657 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 01 16:30:42.924657 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 16:30:42.924657 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 16:30:42.924657 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 16:30:42.924657 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 16:30:42.924657 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 16:30:42.924657 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 16:30:42.924657 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 16:30:42.924657 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application with Lock( Jul 01 16:30:42.924657 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 16:30:42.924657 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._enter() Jul 01 16:30:42.924657 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 16:30:42.924657 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 16:30:42.924657 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 16:30:42.924657 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.creator() Jul 01 16:30:42.924657 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 16:30:42.924657 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application created_value = creator( Jul 01 16:30:42.926058 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 01 16:30:42.926058 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 01 16:30:42.926058 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 16:30:42.926058 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Jul 01 16:30:42.926058 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 615/1242] 10.222.0.57 () {56 vars in 1108 bytes} [Fri Jul 1 16:30:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 16:30:42.931143 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6cedb8-9959-4159-b392-3d3c5c1edf88 None admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:30:43.044599 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=100 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:30:43.051642 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6cedb8-9959-4159-b392-3d3c5c1edf88 None admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:30:43.098428 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b14935-4c75-4394-89d6-41153dd47351 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:30:43.099220 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b14935-4c75-4394-89d6-41153dd47351 None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:30:43.099561 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b14935-4c75-4394-89d6-41153dd47351 None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:30:43.099963 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b14935-4c75-4394-89d6-41153dd47351 None admin] PATH_INFO: `/v3/domains` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:30:43.100984 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b14935-4c75-4394-89d6-41153dd47351 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80312) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:30:43.101372 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b14935-4c75-4394-89d6-41153dd47351 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:30:43.102965 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b14935-4c75-4394-89d6-41153dd47351 None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:30:43.118416 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 628/1243] 10.222.0.57 () {56 vars in 1109 bytes} [Fri Jul 1 16:30:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:30:43.122971 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44dde3ec-b616-4b34-b719-1c0237200bdd None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:30:43.222377 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:30:43.249138 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44dde3ec-b616-4b34-b719-1c0237200bdd None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:30:43.298482 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e77d88-1682-480d-9ed9-5dd55a881bba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:30:43.299415 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e77d88-1682-480d-9ed9-5dd55a881bba None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:30:43.300622 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e77d88-1682-480d-9ed9-5dd55a881bba None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:30:43.300622 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e77d88-1682-480d-9ed9-5dd55a881bba None admin] PATH_INFO: `/v3/projects/service` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:30:43.307662 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e77d88-1682-480d-9ed9-5dd55a881bba None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:30:43.309256 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e77d88-1682-480d-9ed9-5dd55a881bba None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:30:43.326360 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application [None req-84e77d88-1682-480d-9ed9-5dd55a881bba None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 01 16:30:43.326360 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 16:30:43.326360 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 16:30:43.326360 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 16:30:43.326360 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 16:30:43.326360 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 16:30:43.326360 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 16:30:43.326360 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 16:30:43.326360 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 16:30:43.326360 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 16:30:43.326360 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 16:30:43.326360 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 16:30:43.326360 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 01 16:30:43.326360 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 01 16:30:43.326360 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 01 16:30:43.327436 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 01 16:30:43.327436 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 16:30:43.327436 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 16:30:43.327436 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 16:30:43.327436 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 16:30:43.327436 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 16:30:43.327436 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 16:30:43.327436 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 16:30:43.327436 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application with Lock( Jul 01 16:30:43.327436 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 16:30:43.327436 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._enter() Jul 01 16:30:43.327436 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 16:30:43.327436 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 16:30:43.327436 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 16:30:43.327436 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.creator() Jul 01 16:30:43.327436 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 16:30:43.327436 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application created_value = creator( Jul 01 16:30:43.328403 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 01 16:30:43.328403 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 01 16:30:43.328403 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 01 16:30:43.328403 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 01 16:30:43.328403 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 01 16:30:43.328403 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 01 16:30:43.328403 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 01 16:30:43.328403 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Jul 01 16:30:43.328403 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 616/1244] 10.222.0.57 () {56 vars in 1111 bytes} [Fri Jul 1 16:30:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 16:30:43.333713 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b14935-4c75-4394-89d6-41153dd47351 None admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:30:43.471142 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b14935-4c75-4394-89d6-41153dd47351 None admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:30:43.517558 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919f7b5f-8e73-41f7-a025-8647a9658ebb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:30:43.518273 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919f7b5f-8e73-41f7-a025-8647a9658ebb None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:30:43.518703 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919f7b5f-8e73-41f7-a025-8647a9658ebb None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:30:43.518963 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919f7b5f-8e73-41f7-a025-8647a9658ebb None admin] PATH_INFO: `/v3/projects` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:30:43.519767 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-919f7b5f-8e73-41f7-a025-8647a9658ebb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80312) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:30:43.520080 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-919f7b5f-8e73-41f7-a025-8647a9658ebb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:30:43.521757 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-919f7b5f-8e73-41f7-a025-8647a9658ebb None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:30:43.540343 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 629/1245] 10.222.0.57 () {56 vars in 1148 bytes} [Fri Jul 1 16:30:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:30:43.548112 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e77d88-1682-480d-9ed9-5dd55a881bba None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:30:43.703026 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e77d88-1682-480d-9ed9-5dd55a881bba None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:30:43.752572 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15822449-3c00-4771-a993-8a597acc0234 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:30:43.753709 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15822449-3c00-4771-a993-8a597acc0234 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:30:43.755204 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15822449-3c00-4771-a993-8a597acc0234 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:30:43.755204 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15822449-3c00-4771-a993-8a597acc0234 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:30:43.756953 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15822449-3c00-4771-a993-8a597acc0234 None admin] RBAC: Adding query filter params (role.id=e8809b67c9dc471daa1c2bd42c9a2baf, scope.project.id=e0788e6725a84e57a4174284c1845755, user.id=bbc38af18dec4e9b8143df23af6aad43) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:30:43.757656 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15822449-3c00-4771-a993-8a597acc0234 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:30:43.760237 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15822449-3c00-4771-a993-8a597acc0234 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:30:43.772000 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 617/1246] 10.222.0.57 () {56 vars in 1374 bytes} [Fri Jul 1 16:30:43 2022] GET /identity/v3/role_assignments?user.id=bbc38af18dec4e9b8143df23af6aad43&scope.project.id=e0788e6725a84e57a4174284c1845755&role.id=e8809b67c9dc471daa1c2bd42c9a2baf => generated 608 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:30:46.153937 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da69d2fb-65ae-430a-ac4f-0b897e10fd26 None None] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:30:46.154643 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da69d2fb-65ae-430a-ac4f-0b897e10fd26 None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:30:46.154643 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da69d2fb-65ae-430a-ac4f-0b897e10fd26 None None] PATH_INFO: `/` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:30:46.155960 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 630/1247] 10.222.0.57 () {54 vars in 926 bytes} [Fri Jul 1 16:30:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 16:30:46.161222 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63e3075-a895-4c8c-9220-f70a9c842364 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:30:46.161756 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63e3075-a895-4c8c-9220-f70a9c842364 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:30:46.161756 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63e3075-a895-4c8c-9220-f70a9c842364 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:30:46.167938 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:30:46.184037 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-b63e3075-a895-4c8c-9220-f70a9c842364 None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:30:46.185480 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-b63e3075-a895-4c8c-9220-f70a9c842364 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:30:46.277076 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 618/1248] 10.222.0.57 () {58 vars in 1023 bytes} [Fri Jul 1 16:30:46 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 01 16:30:46.298188 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da69d2fb-65ae-430a-ac4f-0b897e10fd26 None None] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:30:46.323734 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=100 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:30:46.436553 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da69d2fb-65ae-430a-ac4f-0b897e10fd26 None None] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:30:46.479977 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a293a0-a39e-4e5b-9d5d-29329f6960d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:30:46.481038 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a293a0-a39e-4e5b-9d5d-29329f6960d3 None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:30:46.481364 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a293a0-a39e-4e5b-9d5d-29329f6960d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:30:46.481681 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a293a0-a39e-4e5b-9d5d-29329f6960d3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:30:46.494598 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a293a0-a39e-4e5b-9d5d-29329f6960d3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 01 16:30:46.503610 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application [None req-10a293a0-a39e-4e5b-9d5d-29329f6960d3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 01 16:30:46.503610 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 16:30:46.503610 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 16:30:46.503610 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 16:30:46.503610 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 16:30:46.503610 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 16:30:46.503610 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 16:30:46.503610 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 16:30:46.503610 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 16:30:46.503610 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 16:30:46.503610 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 16:30:46.503610 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 16:30:46.503610 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 01 16:30:46.503610 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 01 16:30:46.503610 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 01 16:30:46.503610 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 01 16:30:46.504703 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 01 16:30:46.504703 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 01 16:30:46.504703 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 01 16:30:46.504703 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 01 16:30:46.504703 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 16:30:46.504703 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 16:30:46.504703 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 16:30:46.504703 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 16:30:46.504703 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 16:30:46.504703 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 16:30:46.504703 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 16:30:46.504703 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application with Lock( Jul 01 16:30:46.504703 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 16:30:46.504703 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self._enter() Jul 01 16:30:46.504703 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 16:30:46.504703 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 16:30:46.505852 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 16:30:46.505852 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.creator() Jul 01 16:30:46.505852 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 16:30:46.505852 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application created_value = creator( Jul 01 16:30:46.505852 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 01 16:30:46.505852 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 01 16:30:46.505852 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 01 16:30:46.505852 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 01 16:30:46.505852 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 01 16:30:46.505852 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 01 16:30:46.505852 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 01 16:30:46.505852 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application Jul 01 16:30:46.505852 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 631/1249] 10.222.0.57 () {56 vars in 1096 bytes} [Fri Jul 1 16:30:46 2022] GET /identity/v3/roles/admin => generated 83 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 16:30:46.512054 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63e3075-a895-4c8c-9220-f70a9c842364 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:30:46.657401 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63e3075-a895-4c8c-9220-f70a9c842364 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:30:46.693617 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:30:46.699482 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1659f9ab-74a7-4022-93e5-9daad4b1bd72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:30:46.700163 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1659f9ab-74a7-4022-93e5-9daad4b1bd72 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:30:46.700447 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1659f9ab-74a7-4022-93e5-9daad4b1bd72 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:30:46.700735 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1659f9ab-74a7-4022-93e5-9daad4b1bd72 None admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:30:46.701491 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1659f9ab-74a7-4022-93e5-9daad4b1bd72 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:30:46.701807 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1659f9ab-74a7-4022-93e5-9daad4b1bd72 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:30:46.703322 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1659f9ab-74a7-4022-93e5-9daad4b1bd72 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:30:46.712510 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 619/1250] 10.222.0.57 () {56 vars in 1099 bytes} [Fri Jul 1 16:30:46 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:30:46.718979 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a293a0-a39e-4e5b-9d5d-29329f6960d3 None admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:30:46.816973 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=100 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:30:46.845415 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a293a0-a39e-4e5b-9d5d-29329f6960d3 None admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:30:46.892812 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a059e3-f7ab-4eab-b5f3-25db42d42af6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:30:46.893617 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a059e3-f7ab-4eab-b5f3-25db42d42af6 None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:30:46.893900 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a059e3-f7ab-4eab-b5f3-25db42d42af6 None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:30:46.894175 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a059e3-f7ab-4eab-b5f3-25db42d42af6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:30:46.907118 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a059e3-f7ab-4eab-b5f3-25db42d42af6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:30:46.908580 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a059e3-f7ab-4eab-b5f3-25db42d42af6 None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:30:46.931699 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application [None req-88a059e3-f7ab-4eab-b5f3-25db42d42af6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 16:30:46.931699 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 16:30:46.931699 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 16:30:46.931699 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 16:30:46.931699 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 16:30:46.931699 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 16:30:46.931699 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 16:30:46.931699 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 16:30:46.931699 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 16:30:46.931699 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 16:30:46.931699 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 16:30:46.931699 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 16:30:46.931699 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 01 16:30:46.931699 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 01 16:30:46.931699 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 01 16:30:46.932752 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 01 16:30:46.932752 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 16:30:46.932752 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 16:30:46.932752 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 16:30:46.932752 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 16:30:46.932752 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 16:30:46.932752 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 16:30:46.932752 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 16:30:46.932752 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application with Lock( Jul 01 16:30:46.932752 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 16:30:46.932752 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self._enter() Jul 01 16:30:46.932752 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 16:30:46.932752 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 16:30:46.932752 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 16:30:46.932752 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.creator() Jul 01 16:30:46.932752 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 16:30:46.932752 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application created_value = creator( Jul 01 16:30:46.933653 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 01 16:30:46.933653 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 01 16:30:46.933653 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 16:30:46.933653 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application Jul 01 16:30:46.933653 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 632/1251] 10.222.0.57 () {56 vars in 1108 bytes} [Fri Jul 1 16:30:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 16:30:46.939528 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1659f9ab-74a7-4022-93e5-9daad4b1bd72 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:30:47.068740 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1659f9ab-74a7-4022-93e5-9daad4b1bd72 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:30:47.113625 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917f71c9-3349-4dec-8907-352bd5456baf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:30:47.114758 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917f71c9-3349-4dec-8907-352bd5456baf None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:30:47.114758 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917f71c9-3349-4dec-8907-352bd5456baf None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:30:47.114960 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917f71c9-3349-4dec-8907-352bd5456baf None admin] PATH_INFO: `/v3/domains` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:30:47.115609 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-917f71c9-3349-4dec-8907-352bd5456baf None admin] RBAC: Adding query filter params (name=Default) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:30:47.115918 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-917f71c9-3349-4dec-8907-352bd5456baf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:30:47.117289 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-917f71c9-3349-4dec-8907-352bd5456baf None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:30:47.130765 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 620/1252] 10.222.0.57 () {56 vars in 1109 bytes} [Fri Jul 1 16:30:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:30:47.136999 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a059e3-f7ab-4eab-b5f3-25db42d42af6 None admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:30:47.253194 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a059e3-f7ab-4eab-b5f3-25db42d42af6 None admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:30:47.293810 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=100 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:30:47.307727 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331a5d1a-495f-4ab8-8b6e-9226d2e1cf3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:30:47.309033 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331a5d1a-495f-4ab8-8b6e-9226d2e1cf3e None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:30:47.309499 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331a5d1a-495f-4ab8-8b6e-9226d2e1cf3e None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:30:47.309955 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331a5d1a-495f-4ab8-8b6e-9226d2e1cf3e None admin] PATH_INFO: `/v3/users/placement` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:30:47.324797 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-331a5d1a-495f-4ab8-8b6e-9226d2e1cf3e None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:30:47.326190 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-331a5d1a-495f-4ab8-8b6e-9226d2e1cf3e None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:30:47.345334 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application [None req-331a5d1a-495f-4ab8-8b6e-9226d2e1cf3e None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 01 16:30:47.345334 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 16:30:47.345334 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 16:30:47.345334 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 16:30:47.345334 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 16:30:47.345334 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 16:30:47.345334 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 16:30:47.345334 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 16:30:47.345334 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 16:30:47.345334 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 16:30:47.345334 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 16:30:47.345334 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 16:30:47.345334 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 01 16:30:47.345334 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 01 16:30:47.345334 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 01 16:30:47.345334 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 01 16:30:47.347375 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 16:30:47.347375 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 16:30:47.347375 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 01 16:30:47.347375 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 01 16:30:47.347375 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 01 16:30:47.347375 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 01 16:30:47.347375 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 16:30:47.347375 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 16:30:47.347375 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 16:30:47.347375 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 16:30:47.347375 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 16:30:47.347375 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application with Lock( Jul 01 16:30:47.347375 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 16:30:47.347375 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self._enter() Jul 01 16:30:47.347375 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 16:30:47.347375 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 16:30:47.347375 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 16:30:47.347375 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.creator() Jul 01 16:30:47.348854 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 16:30:47.348854 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application created_value = creator( Jul 01 16:30:47.348854 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 01 16:30:47.348854 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 01 16:30:47.348854 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 01 16:30:47.348854 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 01 16:30:47.348854 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 01 16:30:47.348854 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 01 16:30:47.348854 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 01 16:30:47.348854 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application Jul 01 16:30:47.348854 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 633/1253] 10.222.0.57 () {56 vars in 1108 bytes} [Fri Jul 1 16:30:47 2022] GET /identity/v3/users/placement => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 16:30:47.356592 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917f71c9-3349-4dec-8907-352bd5456baf None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:30:47.437072 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:30:47.499756 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917f71c9-3349-4dec-8907-352bd5456baf None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:30:47.576535 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177c45e9-7d81-463e-87f6-8e3a8fefb776 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:30:47.577246 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177c45e9-7d81-463e-87f6-8e3a8fefb776 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:30:47.577689 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177c45e9-7d81-463e-87f6-8e3a8fefb776 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:30:47.577926 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177c45e9-7d81-463e-87f6-8e3a8fefb776 None admin] PATH_INFO: `/v3/users` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:30:47.579416 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-177c45e9-7d81-463e-87f6-8e3a8fefb776 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:30:47.580120 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-177c45e9-7d81-463e-87f6-8e3a8fefb776 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:30:47.582095 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-177c45e9-7d81-463e-87f6-8e3a8fefb776 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:30:47.611421 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 621/1254] 10.222.0.57 () {56 vars in 1143 bytes} [Fri Jul 1 16:30:47 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:30:47.617416 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331a5d1a-495f-4ab8-8b6e-9226d2e1cf3e None admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:30:47.797116 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331a5d1a-495f-4ab8-8b6e-9226d2e1cf3e None admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:30:47.846562 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107d8a32-0766-4f1e-862f-9c5e43be3a4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:30:47.847089 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107d8a32-0766-4f1e-862f-9c5e43be3a4e None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:30:47.847408 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107d8a32-0766-4f1e-862f-9c5e43be3a4e None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:30:47.847734 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107d8a32-0766-4f1e-862f-9c5e43be3a4e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:30:47.930076 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=100 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:30:47.951271 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-107d8a32-0766-4f1e-862f-9c5e43be3a4e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:30:47.953102 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-107d8a32-0766-4f1e-862f-9c5e43be3a4e None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:30:48.103411 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 634/1255] 10.222.0.57 () {58 vars in 1282 bytes} [Fri Jul 1 16:30:47 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 486 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 01 16:30:48.109321 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177c45e9-7d81-463e-87f6-8e3a8fefb776 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:30:48.260664 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177c45e9-7d81-463e-87f6-8e3a8fefb776 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:30:48.275689 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:30:48.304040 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33accd53-924c-4d5e-a1a2-8857fccc9ce0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:30:48.304935 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33accd53-924c-4d5e-a1a2-8857fccc9ce0 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:30:48.305223 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33accd53-924c-4d5e-a1a2-8857fccc9ce0 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:30:48.305501 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33accd53-924c-4d5e-a1a2-8857fccc9ce0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:30:48.318281 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33accd53-924c-4d5e-a1a2-8857fccc9ce0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:30:48.320912 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33accd53-924c-4d5e-a1a2-8857fccc9ce0 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:30:48.352436 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application [None req-33accd53-924c-4d5e-a1a2-8857fccc9ce0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 16:30:48.352436 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 16:30:48.352436 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 16:30:48.352436 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 16:30:48.352436 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 16:30:48.352436 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 16:30:48.352436 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 16:30:48.352436 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 16:30:48.352436 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 16:30:48.352436 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 16:30:48.352436 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 16:30:48.352436 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 16:30:48.352436 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 01 16:30:48.352436 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 01 16:30:48.352436 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 01 16:30:48.354322 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 01 16:30:48.354322 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 16:30:48.354322 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 16:30:48.354322 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 16:30:48.354322 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 16:30:48.354322 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 16:30:48.354322 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 16:30:48.354322 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 16:30:48.354322 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application with Lock( Jul 01 16:30:48.354322 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 16:30:48.354322 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._enter() Jul 01 16:30:48.354322 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 16:30:48.354322 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 16:30:48.354322 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 16:30:48.354322 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.creator() Jul 01 16:30:48.354322 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 16:30:48.354322 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application created_value = creator( Jul 01 16:30:48.356008 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 01 16:30:48.356008 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 01 16:30:48.356008 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 16:30:48.356008 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Jul 01 16:30:48.356008 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 622/1256] 10.222.0.57 () {56 vars in 1108 bytes} [Fri Jul 1 16:30:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 01 16:30:48.362363 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107d8a32-0766-4f1e-862f-9c5e43be3a4e None admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:30:48.474710 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=100 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:30:48.499635 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107d8a32-0766-4f1e-862f-9c5e43be3a4e None admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:30:48.548493 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09029438-449f-4717-ad56-eea1eacf51a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:30:48.549427 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09029438-449f-4717-ad56-eea1eacf51a3 None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:30:48.549826 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09029438-449f-4717-ad56-eea1eacf51a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:30:48.550210 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09029438-449f-4717-ad56-eea1eacf51a3 None admin] PATH_INFO: `/v3/domains` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:30:48.551312 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09029438-449f-4717-ad56-eea1eacf51a3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80312) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:30:48.551764 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09029438-449f-4717-ad56-eea1eacf51a3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:30:48.553762 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09029438-449f-4717-ad56-eea1eacf51a3 None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:30:48.570731 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 635/1257] 10.222.0.57 () {56 vars in 1109 bytes} [Fri Jul 1 16:30:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:30:48.579071 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33accd53-924c-4d5e-a1a2-8857fccc9ce0 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:30:48.725142 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33accd53-924c-4d5e-a1a2-8857fccc9ce0 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:30:48.782611 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad4a2b3-71c8-44d9-aef5-3c078f5d0a74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:30:48.788183 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad4a2b3-71c8-44d9-aef5-3c078f5d0a74 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:30:48.788645 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad4a2b3-71c8-44d9-aef5-3c078f5d0a74 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:30:48.789106 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad4a2b3-71c8-44d9-aef5-3c078f5d0a74 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:30:48.812470 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad4a2b3-71c8-44d9-aef5-3c078f5d0a74 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:30:48.814064 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad4a2b3-71c8-44d9-aef5-3c078f5d0a74 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:30:48.834539 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application [None req-4ad4a2b3-71c8-44d9-aef5-3c078f5d0a74 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 01 16:30:48.834539 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 16:30:48.834539 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 16:30:48.834539 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 16:30:48.834539 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 16:30:48.834539 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 16:30:48.834539 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 16:30:48.834539 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 16:30:48.834539 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 16:30:48.834539 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 16:30:48.834539 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 16:30:48.834539 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 16:30:48.834539 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 01 16:30:48.834539 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 01 16:30:48.834539 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 01 16:30:48.836379 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 01 16:30:48.836379 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 16:30:48.836379 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 16:30:48.836379 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 16:30:48.836379 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 16:30:48.836379 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 16:30:48.836379 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 16:30:48.836379 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 16:30:48.836379 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application with Lock( Jul 01 16:30:48.836379 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 16:30:48.836379 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._enter() Jul 01 16:30:48.836379 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 16:30:48.836379 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 16:30:48.836379 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 16:30:48.836379 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.creator() Jul 01 16:30:48.836379 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 16:30:48.836379 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application created_value = creator( Jul 01 16:30:48.843072 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 01 16:30:48.843072 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 01 16:30:48.843072 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 01 16:30:48.843072 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 01 16:30:48.843072 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 01 16:30:48.843072 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 01 16:30:48.843072 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 01 16:30:48.843072 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Jul 01 16:30:48.843072 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 623/1258] 10.222.0.57 () {56 vars in 1111 bytes} [Fri Jul 1 16:30:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 16:30:48.843980 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09029438-449f-4717-ad56-eea1eacf51a3 None admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:30:48.981066 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09029438-449f-4717-ad56-eea1eacf51a3 None admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:30:49.045402 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd941ecf-bdda-4fd5-b4cc-10936162d655 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:30:49.046217 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd941ecf-bdda-4fd5-b4cc-10936162d655 None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:30:49.046559 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd941ecf-bdda-4fd5-b4cc-10936162d655 None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:30:49.047061 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd941ecf-bdda-4fd5-b4cc-10936162d655 None admin] PATH_INFO: `/v3/projects` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:30:49.048132 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd941ecf-bdda-4fd5-b4cc-10936162d655 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80312) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:30:49.048456 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd941ecf-bdda-4fd5-b4cc-10936162d655 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:30:49.050531 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd941ecf-bdda-4fd5-b4cc-10936162d655 None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:30:49.066557 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 636/1259] 10.222.0.57 () {56 vars in 1148 bytes} [Fri Jul 1 16:30:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:30:49.074746 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad4a2b3-71c8-44d9-aef5-3c078f5d0a74 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:30:49.118562 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:30:49.273617 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad4a2b3-71c8-44d9-aef5-3c078f5d0a74 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:30:49.342514 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d40e0b-b270-4c27-9384-202184c8b24d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:30:49.343429 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d40e0b-b270-4c27-9384-202184c8b24d None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:30:49.343652 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d40e0b-b270-4c27-9384-202184c8b24d None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:30:49.343950 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d40e0b-b270-4c27-9384-202184c8b24d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:30:49.344820 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d40e0b-b270-4c27-9384-202184c8b24d None admin] RBAC: Adding query filter params (role.id=84ee739146934c1ab32c501ff8093644, scope.project.id=e0788e6725a84e57a4174284c1845755, user.id=bbc38af18dec4e9b8143df23af6aad43) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:30:49.345149 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d40e0b-b270-4c27-9384-202184c8b24d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:30:49.347628 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d40e0b-b270-4c27-9384-202184c8b24d None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:30:49.355311 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 624/1260] 10.222.0.57 () {56 vars in 1374 bytes} [Fri Jul 1 16:30:49 2022] GET /identity/v3/role_assignments?user.id=bbc38af18dec4e9b8143df23af6aad43&scope.project.id=e0788e6725a84e57a4174284c1845755&role.id=84ee739146934c1ab32c501ff8093644 => generated 259 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:30:51.684020 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8db937c-8b40-41a3-ac1e-2bbc883b8b65 None None] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:30:51.684560 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8db937c-8b40-41a3-ac1e-2bbc883b8b65 None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:30:51.684560 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8db937c-8b40-41a3-ac1e-2bbc883b8b65 None None] PATH_INFO: `/` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:30:51.685948 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 637/1261] 10.222.0.57 () {54 vars in 926 bytes} [Fri Jul 1 16:30:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 16:30:51.691328 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bfb5a6-9a27-4deb-97bc-815199b6c694 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:30:51.691723 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bfb5a6-9a27-4deb-97bc-815199b6c694 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:30:51.691894 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bfb5a6-9a27-4deb-97bc-815199b6c694 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:30:51.709167 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-71bfb5a6-9a27-4deb-97bc-815199b6c694 None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:30:51.710389 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-71bfb5a6-9a27-4deb-97bc-815199b6c694 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:30:51.781912 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 625/1262] 10.222.0.57 () {58 vars in 1023 bytes} [Fri Jul 1 16:30:51 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 01 16:30:51.804849 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8db937c-8b40-41a3-ac1e-2bbc883b8b65 None None] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:30:51.815712 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=100 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:30:51.911798 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8db937c-8b40-41a3-ac1e-2bbc883b8b65 None None] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:30:51.945160 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d91406-b8cf-40cc-bd23-d1bf5eea51cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:30:51.945869 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d91406-b8cf-40cc-bd23-d1bf5eea51cc None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:30:51.946151 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d91406-b8cf-40cc-bd23-d1bf5eea51cc None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:30:51.946460 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d91406-b8cf-40cc-bd23-d1bf5eea51cc None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:30:51.960136 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d91406-b8cf-40cc-bd23-d1bf5eea51cc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 01 16:30:51.968550 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application [None req-b2d91406-b8cf-40cc-bd23-d1bf5eea51cc None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 01 16:30:51.968550 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 16:30:51.968550 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 16:30:51.968550 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 16:30:51.968550 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 16:30:51.968550 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 16:30:51.968550 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 16:30:51.968550 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 16:30:51.968550 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 16:30:51.968550 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 16:30:51.968550 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 16:30:51.968550 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 16:30:51.968550 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 01 16:30:51.968550 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 01 16:30:51.968550 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 01 16:30:51.968550 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 01 16:30:51.970122 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 01 16:30:51.970122 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 01 16:30:51.970122 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 01 16:30:51.970122 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 01 16:30:51.970122 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 16:30:51.970122 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 16:30:51.970122 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 16:30:51.970122 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 16:30:51.970122 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 16:30:51.970122 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 16:30:51.970122 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 16:30:51.970122 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application with Lock( Jul 01 16:30:51.970122 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 16:30:51.970122 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self._enter() Jul 01 16:30:51.970122 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 16:30:51.970122 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 16:30:51.971816 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 16:30:51.971816 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.creator() Jul 01 16:30:51.971816 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 16:30:51.971816 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application created_value = creator( Jul 01 16:30:51.971816 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 01 16:30:51.971816 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 01 16:30:51.971816 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 01 16:30:51.971816 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 01 16:30:51.971816 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 01 16:30:51.971816 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 01 16:30:51.971816 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 01 16:30:51.971816 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application Jul 01 16:30:51.971816 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 638/1263] 10.222.0.57 () {56 vars in 1096 bytes} [Fri Jul 1 16:30:51 2022] GET /identity/v3/roles/admin => generated 83 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 16:30:51.977545 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71bfb5a6-9a27-4deb-97bc-815199b6c694 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:30:51.997581 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:30:52.083581 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71bfb5a6-9a27-4deb-97bc-815199b6c694 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:30:52.122980 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98fbbc62-1b72-4d92-977b-26ce66347663 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:30:52.123655 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fbbc62-1b72-4d92-977b-26ce66347663 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:30:52.123938 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fbbc62-1b72-4d92-977b-26ce66347663 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:30:52.124214 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fbbc62-1b72-4d92-977b-26ce66347663 None admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:30:52.124965 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98fbbc62-1b72-4d92-977b-26ce66347663 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:30:52.125275 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98fbbc62-1b72-4d92-977b-26ce66347663 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:30:52.126662 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98fbbc62-1b72-4d92-977b-26ce66347663 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:30:52.136550 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 626/1264] 10.222.0.57 () {56 vars in 1099 bytes} [Fri Jul 1 16:30:51 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:30:52.142549 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d91406-b8cf-40cc-bd23-d1bf5eea51cc None admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:30:52.211665 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=100 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:30:52.245187 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d91406-b8cf-40cc-bd23-d1bf5eea51cc None admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:30:52.278302 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf4c741-5780-456a-9094-50d2117b77eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:30:52.279121 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf4c741-5780-456a-9094-50d2117b77eb None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:30:52.279403 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf4c741-5780-456a-9094-50d2117b77eb None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:30:52.279677 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf4c741-5780-456a-9094-50d2117b77eb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:30:52.289895 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf4c741-5780-456a-9094-50d2117b77eb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:30:52.291541 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf4c741-5780-456a-9094-50d2117b77eb None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:30:52.309311 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application [None req-5bf4c741-5780-456a-9094-50d2117b77eb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 16:30:52.309311 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 16:30:52.309311 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 16:30:52.309311 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 16:30:52.309311 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 16:30:52.309311 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 16:30:52.309311 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 16:30:52.309311 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 16:30:52.309311 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 16:30:52.309311 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 16:30:52.309311 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 16:30:52.309311 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 16:30:52.309311 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 01 16:30:52.309311 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 01 16:30:52.309311 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 01 16:30:52.310469 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 01 16:30:52.310469 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 16:30:52.310469 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 16:30:52.310469 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 16:30:52.310469 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 16:30:52.310469 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 16:30:52.310469 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 16:30:52.310469 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 16:30:52.310469 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application with Lock( Jul 01 16:30:52.310469 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 16:30:52.310469 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self._enter() Jul 01 16:30:52.310469 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 16:30:52.310469 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 16:30:52.310469 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 16:30:52.310469 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.creator() Jul 01 16:30:52.310469 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 16:30:52.310469 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application created_value = creator( Jul 01 16:30:52.311577 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 01 16:30:52.311577 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 01 16:30:52.311577 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 16:30:52.311577 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application Jul 01 16:30:52.311577 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 639/1265] 10.222.0.57 () {56 vars in 1108 bytes} [Fri Jul 1 16:30:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 16:30:52.316470 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98fbbc62-1b72-4d92-977b-26ce66347663 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:30:52.397438 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:30:52.417782 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98fbbc62-1b72-4d92-977b-26ce66347663 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:30:52.450876 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5981f4-954c-4bd6-a62d-600bba51f961 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:30:52.451529 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5981f4-954c-4bd6-a62d-600bba51f961 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:30:52.451814 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5981f4-954c-4bd6-a62d-600bba51f961 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:30:52.452094 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5981f4-954c-4bd6-a62d-600bba51f961 None admin] PATH_INFO: `/v3/domains` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:30:52.452809 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be5981f4-954c-4bd6-a62d-600bba51f961 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:30:52.453116 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be5981f4-954c-4bd6-a62d-600bba51f961 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:30:52.454499 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be5981f4-954c-4bd6-a62d-600bba51f961 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:30:52.471040 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 627/1266] 10.222.0.57 () {56 vars in 1109 bytes} [Fri Jul 1 16:30:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:30:52.476829 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf4c741-5780-456a-9094-50d2117b77eb None admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:30:52.607069 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf4c741-5780-456a-9094-50d2117b77eb None admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:30:52.636191 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=100 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:30:52.651745 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112d130e-797c-426c-b7d9-81dc9e4f8e7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:30:52.652642 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112d130e-797c-426c-b7d9-81dc9e4f8e7d None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:30:52.652967 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112d130e-797c-426c-b7d9-81dc9e4f8e7d None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:30:52.653252 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112d130e-797c-426c-b7d9-81dc9e4f8e7d None admin] PATH_INFO: `/v3/users/placement` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:30:52.660723 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-112d130e-797c-426c-b7d9-81dc9e4f8e7d None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:30:52.662162 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-112d130e-797c-426c-b7d9-81dc9e4f8e7d None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:30:52.676359 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application [None req-112d130e-797c-426c-b7d9-81dc9e4f8e7d None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 01 16:30:52.676359 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 16:30:52.676359 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 16:30:52.676359 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 16:30:52.676359 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 16:30:52.676359 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 16:30:52.676359 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 16:30:52.676359 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 16:30:52.676359 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 16:30:52.676359 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 16:30:52.676359 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 16:30:52.676359 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 16:30:52.676359 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 01 16:30:52.676359 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 01 16:30:52.676359 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 01 16:30:52.676359 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 01 16:30:52.677464 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 16:30:52.677464 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 16:30:52.677464 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 01 16:30:52.677464 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 01 16:30:52.677464 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 01 16:30:52.677464 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 01 16:30:52.677464 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 16:30:52.677464 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 16:30:52.677464 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 16:30:52.677464 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 16:30:52.677464 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 16:30:52.677464 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application with Lock( Jul 01 16:30:52.677464 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 16:30:52.677464 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self._enter() Jul 01 16:30:52.677464 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 16:30:52.677464 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 16:30:52.677464 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 16:30:52.677464 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.creator() Jul 01 16:30:52.678477 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 16:30:52.678477 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application created_value = creator( Jul 01 16:30:52.678477 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 01 16:30:52.678477 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 01 16:30:52.678477 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 01 16:30:52.678477 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 01 16:30:52.678477 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 01 16:30:52.678477 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 01 16:30:52.678477 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 01 16:30:52.678477 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application Jul 01 16:30:52.678477 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 640/1267] 10.222.0.57 () {56 vars in 1108 bytes} [Fri Jul 1 16:30:52 2022] GET /identity/v3/users/placement => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 16:30:52.683537 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5981f4-954c-4bd6-a62d-600bba51f961 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:30:52.783501 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5981f4-954c-4bd6-a62d-600bba51f961 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:30:52.829154 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428cd450-6422-4e70-bae1-8170daed1f0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:30:52.829824 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428cd450-6422-4e70-bae1-8170daed1f0d None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:30:52.830110 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428cd450-6422-4e70-bae1-8170daed1f0d None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:30:52.830415 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428cd450-6422-4e70-bae1-8170daed1f0d None admin] PATH_INFO: `/v3/users` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:30:52.831259 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-428cd450-6422-4e70-bae1-8170daed1f0d None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:30:52.831604 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-428cd450-6422-4e70-bae1-8170daed1f0d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:30:52.833055 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-428cd450-6422-4e70-bae1-8170daed1f0d None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:30:52.842554 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:30:52.852402 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 628/1268] 10.222.0.57 () {56 vars in 1143 bytes} [Fri Jul 1 16:30:52 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:30:52.858974 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112d130e-797c-426c-b7d9-81dc9e4f8e7d None admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:30:52.961611 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112d130e-797c-426c-b7d9-81dc9e4f8e7d None admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:30:52.995118 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e0d0fc-7ade-4584-a0c9-f06641ad69a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:30:52.995920 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e0d0fc-7ade-4584-a0c9-f06641ad69a6 None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:30:52.996206 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e0d0fc-7ade-4584-a0c9-f06641ad69a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:30:52.996482 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e0d0fc-7ade-4584-a0c9-f06641ad69a6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:30:53.006741 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e0d0fc-7ade-4584-a0c9-f06641ad69a6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:30:53.008177 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e0d0fc-7ade-4584-a0c9-f06641ad69a6 None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:30:53.026105 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application [None req-d3e0d0fc-7ade-4584-a0c9-f06641ad69a6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 16:30:53.026105 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 16:30:53.026105 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 16:30:53.026105 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 16:30:53.026105 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 16:30:53.026105 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 16:30:53.026105 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 16:30:53.026105 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 16:30:53.026105 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 16:30:53.026105 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 16:30:53.026105 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 16:30:53.026105 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 16:30:53.026105 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 01 16:30:53.026105 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 01 16:30:53.026105 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 01 16:30:53.027193 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 01 16:30:53.027193 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 16:30:53.027193 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 16:30:53.027193 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 16:30:53.027193 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 16:30:53.027193 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 16:30:53.027193 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 16:30:53.027193 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 16:30:53.027193 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application with Lock( Jul 01 16:30:53.027193 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 16:30:53.027193 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self._enter() Jul 01 16:30:53.027193 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 16:30:53.027193 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 16:30:53.027193 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 16:30:53.027193 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.creator() Jul 01 16:30:53.027193 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 16:30:53.027193 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application created_value = creator( Jul 01 16:30:53.028309 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 01 16:30:53.028309 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 01 16:30:53.028309 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 16:30:53.028309 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application Jul 01 16:30:53.028309 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 641/1269] 10.222.0.57 () {56 vars in 1108 bytes} [Fri Jul 1 16:30:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 16:30:53.033072 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428cd450-6422-4e70-bae1-8170daed1f0d None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:30:53.132215 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428cd450-6422-4e70-bae1-8170daed1f0d None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:30:53.175551 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed053c75-cc12-4e5b-a009-66a2f631ad11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:30:53.176220 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed053c75-cc12-4e5b-a009-66a2f631ad11 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:30:53.176514 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed053c75-cc12-4e5b-a009-66a2f631ad11 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:30:53.176797 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed053c75-cc12-4e5b-a009-66a2f631ad11 None admin] PATH_INFO: `/v3/domains` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:30:53.177529 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed053c75-cc12-4e5b-a009-66a2f631ad11 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:30:53.177840 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed053c75-cc12-4e5b-a009-66a2f631ad11 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:30:53.179553 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed053c75-cc12-4e5b-a009-66a2f631ad11 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:30:53.193016 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 629/1270] 10.222.0.57 () {56 vars in 1109 bytes} [Fri Jul 1 16:30:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:30:53.199049 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e0d0fc-7ade-4584-a0c9-f06641ad69a6 None admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:30:53.231369 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=100 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:30:53.298460 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e0d0fc-7ade-4584-a0c9-f06641ad69a6 None admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:30:53.331773 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a49329-db9c-4b15-bf6f-e432149b7547 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:30:53.332548 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a49329-db9c-4b15-bf6f-e432149b7547 None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:30:53.332828 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a49329-db9c-4b15-bf6f-e432149b7547 None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:30:53.333102 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a49329-db9c-4b15-bf6f-e432149b7547 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:30:53.339911 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a49329-db9c-4b15-bf6f-e432149b7547 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:30:53.341333 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a49329-db9c-4b15-bf6f-e432149b7547 None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:30:53.356011 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application [None req-e9a49329-db9c-4b15-bf6f-e432149b7547 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 01 16:30:53.356011 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 16:30:53.356011 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 16:30:53.356011 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 16:30:53.356011 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 16:30:53.356011 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 16:30:53.356011 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 16:30:53.356011 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 16:30:53.356011 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 16:30:53.356011 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 16:30:53.356011 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 16:30:53.356011 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 16:30:53.356011 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 01 16:30:53.356011 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 01 16:30:53.356011 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 01 16:30:53.357073 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 01 16:30:53.357073 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 16:30:53.357073 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 16:30:53.357073 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 16:30:53.357073 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 16:30:53.357073 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 16:30:53.357073 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 16:30:53.357073 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 16:30:53.357073 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application with Lock( Jul 01 16:30:53.357073 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 16:30:53.357073 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self._enter() Jul 01 16:30:53.357073 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 16:30:53.357073 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 16:30:53.357073 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 16:30:53.357073 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.creator() Jul 01 16:30:53.357073 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 16:30:53.357073 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application created_value = creator( Jul 01 16:30:53.358132 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 01 16:30:53.358132 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 01 16:30:53.358132 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 01 16:30:53.358132 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 01 16:30:53.358132 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 01 16:30:53.358132 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 01 16:30:53.358132 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 01 16:30:53.358132 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application Jul 01 16:30:53.358132 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 642/1271] 10.222.0.57 () {56 vars in 1111 bytes} [Fri Jul 1 16:30:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 16:30:53.363020 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed053c75-cc12-4e5b-a009-66a2f631ad11 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:30:53.424767 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:30:53.466707 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed053c75-cc12-4e5b-a009-66a2f631ad11 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:30:53.500659 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a51f29-e0c1-4324-aa58-ca7b96bb0832 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:30:53.501778 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a51f29-e0c1-4324-aa58-ca7b96bb0832 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:30:53.502301 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a51f29-e0c1-4324-aa58-ca7b96bb0832 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:30:53.502831 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a51f29-e0c1-4324-aa58-ca7b96bb0832 None admin] PATH_INFO: `/v3/projects` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:30:53.504334 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a51f29-e0c1-4324-aa58-ca7b96bb0832 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:30:53.504883 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a51f29-e0c1-4324-aa58-ca7b96bb0832 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:30:53.507307 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a51f29-e0c1-4324-aa58-ca7b96bb0832 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:30:53.525135 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 630/1272] 10.222.0.57 () {56 vars in 1148 bytes} [Fri Jul 1 16:30:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:30:53.532083 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a49329-db9c-4b15-bf6f-e432149b7547 None admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:30:53.631170 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=100 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:30:53.640392 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a49329-db9c-4b15-bf6f-e432149b7547 None admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:30:53.674237 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6677f488-3f59-4b6f-8a23-504a776f5dbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:30:53.675213 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6677f488-3f59-4b6f-8a23-504a776f5dbc None admin] REQUEST_METHOD: `PUT` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:30:53.675445 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6677f488-3f59-4b6f-8a23-504a776f5dbc None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:30:53.675728 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6677f488-3f59-4b6f-8a23-504a776f5dbc None admin] PATH_INFO: `/v3/projects/e0788e6725a84e57a4174284c1845755/users/bbc38af18dec4e9b8143df23af6aad43/roles/84ee739146934c1ab32c501ff8093644` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:30:53.677694 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6677f488-3f59-4b6f-8a23-504a776f5dbc None admin] RBAC: Authorizing `identity:create_grant(project_id=e0788e6725a84e57a4174284c1845755, user_id=bbc38af18dec4e9b8143df23af6aad43, role_id=84ee739146934c1ab32c501ff8093644)` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:30:53.679154 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6677f488-3f59-4b6f-8a23-504a776f5dbc None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:30:53.692016 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 643/1273] 10.222.0.57 () {58 vars in 1439 bytes} [Fri Jul 1 16:30:53 2022] PUT /identity/v3/projects/e0788e6725a84e57a4174284c1845755/users/bbc38af18dec4e9b8143df23af6aad43/roles/84ee739146934c1ab32c501ff8093644 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 16:30:55.760322 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5538c5a3-00af-44a8-b5ba-56564260a910 None None] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:30:55.760936 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5538c5a3-00af-44a8-b5ba-56564260a910 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:30:55.760936 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5538c5a3-00af-44a8-b5ba-56564260a910 None None] PATH_INFO: `/` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:30:55.762238 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 631/1274] 10.222.0.57 () {54 vars in 926 bytes} [Fri Jul 1 16:30:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 16:30:55.767462 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e124e2-ffc7-4cfe-9f71-a7d1d2f16e63 None None] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:30:55.767865 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e124e2-ffc7-4cfe-9f71-a7d1d2f16e63 None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:30:55.768044 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e124e2-ffc7-4cfe-9f71-a7d1d2f16e63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:30:55.785593 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.auth.core [None req-75e124e2-ffc7-4cfe-9f71-a7d1d2f16e63 None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80312) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:30:55.786817 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-75e124e2-ffc7-4cfe-9f71-a7d1d2f16e63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:30:55.852251 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 644/1275] 10.222.0.57 () {58 vars in 1023 bytes} [Fri Jul 1 16:30:55 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 01 16:30:55.876437 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5538c5a3-00af-44a8-b5ba-56564260a910 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:30:55.973928 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5538c5a3-00af-44a8-b5ba-56564260a910 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:30:55.990392 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:30:56.007382 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a1ab35-19d0-4005-a763-e988c18e7ac7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:30:56.008144 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a1ab35-19d0-4005-a763-e988c18e7ac7 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:30:56.008425 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a1ab35-19d0-4005-a763-e988c18e7ac7 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:30:56.008696 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a1ab35-19d0-4005-a763-e988c18e7ac7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:30:56.019194 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a1ab35-19d0-4005-a763-e988c18e7ac7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 01 16:30:56.026934 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application [None req-89a1ab35-19d0-4005-a763-e988c18e7ac7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 01 16:30:56.026934 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 16:30:56.026934 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 16:30:56.026934 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 16:30:56.026934 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 16:30:56.026934 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 16:30:56.026934 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 16:30:56.026934 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 16:30:56.026934 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 16:30:56.026934 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 16:30:56.026934 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 16:30:56.026934 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 16:30:56.026934 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 01 16:30:56.026934 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 01 16:30:56.026934 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 01 16:30:56.026934 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 01 16:30:56.028166 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 01 16:30:56.028166 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 01 16:30:56.028166 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 01 16:30:56.028166 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 01 16:30:56.028166 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 16:30:56.028166 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 16:30:56.028166 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 16:30:56.028166 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 16:30:56.028166 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 16:30:56.028166 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 16:30:56.028166 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 16:30:56.028166 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application with Lock( Jul 01 16:30:56.028166 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 16:30:56.028166 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._enter() Jul 01 16:30:56.028166 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 16:30:56.028166 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 16:30:56.029056 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 16:30:56.029056 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.creator() Jul 01 16:30:56.029056 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 16:30:56.029056 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application created_value = creator( Jul 01 16:30:56.029056 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 01 16:30:56.029056 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 01 16:30:56.029056 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 01 16:30:56.029056 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 01 16:30:56.029056 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 01 16:30:56.029056 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 01 16:30:56.029056 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 01 16:30:56.029056 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Jul 01 16:30:56.029056 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 632/1276] 10.222.0.57 () {56 vars in 1096 bytes} [Fri Jul 1 16:30:55 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 16:30:56.034048 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e124e2-ffc7-4cfe-9f71-a7d1d2f16e63 None None] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:30:56.112705 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:30:56.131700 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e124e2-ffc7-4cfe-9f71-a7d1d2f16e63 None None] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:30:56.164888 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ad4239-ea21-4b23-b721-616ebe8c6f52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:30:56.166096 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ad4239-ea21-4b23-b721-616ebe8c6f52 None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:30:56.166518 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ad4239-ea21-4b23-b721-616ebe8c6f52 None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:30:56.166795 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ad4239-ea21-4b23-b721-616ebe8c6f52 None admin] PATH_INFO: `/v3/roles` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:30:56.167563 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ad4239-ea21-4b23-b721-616ebe8c6f52 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80312) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:30:56.167865 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ad4239-ea21-4b23-b721-616ebe8c6f52 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:30:56.169246 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ad4239-ea21-4b23-b721-616ebe8c6f52 None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:30:56.177220 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 645/1277] 10.222.0.57 () {56 vars in 1099 bytes} [Fri Jul 1 16:30:56 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:30:56.183495 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a1ab35-19d0-4005-a763-e988c18e7ac7 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:30:56.280607 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a1ab35-19d0-4005-a763-e988c18e7ac7 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:30:56.313938 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b80a64-c21f-4cf7-9a8c-a66e404ac5bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:30:56.314743 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b80a64-c21f-4cf7-9a8c-a66e404ac5bd None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:30:56.315016 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b80a64-c21f-4cf7-9a8c-a66e404ac5bd None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:30:56.315287 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b80a64-c21f-4cf7-9a8c-a66e404ac5bd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:30:56.325969 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b80a64-c21f-4cf7-9a8c-a66e404ac5bd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:30:56.327448 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b80a64-c21f-4cf7-9a8c-a66e404ac5bd None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:30:56.343781 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application [None req-d0b80a64-c21f-4cf7-9a8c-a66e404ac5bd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 16:30:56.343781 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 16:30:56.343781 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 16:30:56.343781 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 16:30:56.343781 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 16:30:56.343781 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 16:30:56.343781 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 16:30:56.343781 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 16:30:56.343781 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 16:30:56.343781 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 16:30:56.343781 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 16:30:56.343781 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 16:30:56.343781 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 01 16:30:56.343781 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 01 16:30:56.343781 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 01 16:30:56.344828 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 01 16:30:56.344828 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 16:30:56.344828 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 16:30:56.344828 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 16:30:56.344828 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 16:30:56.344828 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 16:30:56.344828 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 16:30:56.344828 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 16:30:56.344828 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application with Lock( Jul 01 16:30:56.344828 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 16:30:56.344828 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._enter() Jul 01 16:30:56.344828 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 16:30:56.344828 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 16:30:56.344828 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 16:30:56.344828 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.creator() Jul 01 16:30:56.344828 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 16:30:56.344828 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application created_value = creator( Jul 01 16:30:56.346017 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 01 16:30:56.346017 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 01 16:30:56.346017 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 16:30:56.346017 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Jul 01 16:30:56.346017 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 633/1278] 10.222.0.57 () {56 vars in 1108 bytes} [Fri Jul 1 16:30:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 16:30:56.350649 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ad4239-ea21-4b23-b721-616ebe8c6f52 None admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:30:56.452302 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ad4239-ea21-4b23-b721-616ebe8c6f52 None admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:30:56.484888 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d49b61f-0a06-4a8c-930d-cd361187a646 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:30:56.485545 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d49b61f-0a06-4a8c-930d-cd361187a646 None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:30:56.485850 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d49b61f-0a06-4a8c-930d-cd361187a646 None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:30:56.486112 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d49b61f-0a06-4a8c-930d-cd361187a646 None admin] PATH_INFO: `/v3/domains` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:30:56.486863 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d49b61f-0a06-4a8c-930d-cd361187a646 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80312) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:30:56.487176 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d49b61f-0a06-4a8c-930d-cd361187a646 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:30:56.488554 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d49b61f-0a06-4a8c-930d-cd361187a646 None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:30:56.500421 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 646/1279] 10.222.0.57 () {56 vars in 1109 bytes} [Fri Jul 1 16:30:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:30:56.506042 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b80a64-c21f-4cf7-9a8c-a66e404ac5bd None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:30:56.538611 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:30:56.617961 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b80a64-c21f-4cf7-9a8c-a66e404ac5bd None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:30:56.652035 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68504a0-c063-4514-a598-24b7aef06935 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:30:56.653371 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68504a0-c063-4514-a598-24b7aef06935 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:30:56.653722 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68504a0-c063-4514-a598-24b7aef06935 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:30:56.654232 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68504a0-c063-4514-a598-24b7aef06935 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:30:56.663015 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d68504a0-c063-4514-a598-24b7aef06935 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:30:56.664457 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d68504a0-c063-4514-a598-24b7aef06935 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:30:56.679454 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application [None req-d68504a0-c063-4514-a598-24b7aef06935 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 01 16:30:56.679454 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 16:30:56.679454 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 16:30:56.679454 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 16:30:56.679454 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 16:30:56.679454 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 16:30:56.679454 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 16:30:56.679454 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 16:30:56.679454 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 16:30:56.679454 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 16:30:56.679454 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 16:30:56.679454 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 16:30:56.679454 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 01 16:30:56.679454 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 01 16:30:56.679454 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 01 16:30:56.679454 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 01 16:30:56.680588 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 16:30:56.680588 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 16:30:56.680588 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 01 16:30:56.680588 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 01 16:30:56.680588 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 01 16:30:56.680588 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 01 16:30:56.680588 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 16:30:56.680588 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 16:30:56.680588 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 16:30:56.680588 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 16:30:56.680588 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 16:30:56.680588 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application with Lock( Jul 01 16:30:56.680588 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 16:30:56.680588 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._enter() Jul 01 16:30:56.680588 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 16:30:56.680588 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 16:30:56.680588 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 16:30:56.680588 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.creator() Jul 01 16:30:56.681708 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 16:30:56.681708 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application created_value = creator( Jul 01 16:30:56.681708 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 01 16:30:56.681708 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 01 16:30:56.681708 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 01 16:30:56.681708 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 01 16:30:56.681708 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 01 16:30:56.681708 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 01 16:30:56.681708 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 01 16:30:56.681708 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Jul 01 16:30:56.681708 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 634/1280] 10.222.0.57 () {56 vars in 1108 bytes} [Fri Jul 1 16:30:56 2022] GET /identity/v3/users/placement => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 16:30:56.686573 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d49b61f-0a06-4a8c-930d-cd361187a646 None admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:30:56.690495 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=100 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:30:56.784257 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d49b61f-0a06-4a8c-930d-cd361187a646 None admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:30:56.817465 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04882570-cdfc-4e1d-9040-e53c24f2f628 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:30:56.818059 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04882570-cdfc-4e1d-9040-e53c24f2f628 None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:30:56.818360 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04882570-cdfc-4e1d-9040-e53c24f2f628 None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:30:56.818651 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04882570-cdfc-4e1d-9040-e53c24f2f628 None admin] PATH_INFO: `/v3/users` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:30:56.819483 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04882570-cdfc-4e1d-9040-e53c24f2f628 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80312) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:30:56.819795 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04882570-cdfc-4e1d-9040-e53c24f2f628 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:30:56.821217 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04882570-cdfc-4e1d-9040-e53c24f2f628 None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:30:56.841972 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 647/1281] 10.222.0.57 () {56 vars in 1143 bytes} [Fri Jul 1 16:30:56 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:30:56.848104 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68504a0-c063-4514-a598-24b7aef06935 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:30:56.939861 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:30:56.950590 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68504a0-c063-4514-a598-24b7aef06935 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:30:56.983429 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1ab554-6c03-4db3-a82c-8cd3ddec1426 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:30:56.984061 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1ab554-6c03-4db3-a82c-8cd3ddec1426 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:30:56.984345 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1ab554-6c03-4db3-a82c-8cd3ddec1426 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:30:56.984624 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1ab554-6c03-4db3-a82c-8cd3ddec1426 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:30:57.049714 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b1ab554-6c03-4db3-a82c-8cd3ddec1426 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:30:57.051173 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b1ab554-6c03-4db3-a82c-8cd3ddec1426 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:30:57.140314 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:30:57.153001 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 635/1282] 10.222.0.57 () {58 vars in 1282 bytes} [Fri Jul 1 16:30:56 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 01 16:30:57.158839 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04882570-cdfc-4e1d-9040-e53c24f2f628 None admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:30:57.219931 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=100 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:30:57.257893 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04882570-cdfc-4e1d-9040-e53c24f2f628 None admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:30:57.290381 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a16d51-26bf-4fdd-8211-f79a1ad7176c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:30:57.291885 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a16d51-26bf-4fdd-8211-f79a1ad7176c None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:30:57.292174 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a16d51-26bf-4fdd-8211-f79a1ad7176c None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:30:57.292456 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a16d51-26bf-4fdd-8211-f79a1ad7176c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:30:57.302450 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a16d51-26bf-4fdd-8211-f79a1ad7176c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:30:57.303895 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a16d51-26bf-4fdd-8211-f79a1ad7176c None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:30:57.321604 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application [None req-c7a16d51-26bf-4fdd-8211-f79a1ad7176c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 16:30:57.321604 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 16:30:57.321604 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 16:30:57.321604 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 16:30:57.321604 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 16:30:57.321604 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 16:30:57.321604 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 16:30:57.321604 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 16:30:57.321604 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 16:30:57.321604 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 16:30:57.321604 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 16:30:57.321604 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 16:30:57.321604 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 01 16:30:57.321604 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 01 16:30:57.321604 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 01 16:30:57.322956 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 01 16:30:57.322956 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 16:30:57.322956 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 16:30:57.322956 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 16:30:57.322956 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 16:30:57.322956 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 16:30:57.322956 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 16:30:57.322956 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 16:30:57.322956 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application with Lock( Jul 01 16:30:57.322956 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 16:30:57.322956 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self._enter() Jul 01 16:30:57.322956 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 16:30:57.322956 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 16:30:57.322956 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 16:30:57.322956 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.creator() Jul 01 16:30:57.322956 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 16:30:57.322956 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application created_value = creator( Jul 01 16:30:57.324156 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 01 16:30:57.324156 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 01 16:30:57.324156 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 16:30:57.324156 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application Jul 01 16:30:57.324156 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 648/1283] 10.222.0.57 () {56 vars in 1108 bytes} [Fri Jul 1 16:30:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 16:30:57.329052 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1ab554-6c03-4db3-a82c-8cd3ddec1426 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:30:57.425795 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1ab554-6c03-4db3-a82c-8cd3ddec1426 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:30:57.460985 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37640949-2e98-47ef-9c63-65ef59e66bf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:30:57.461634 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37640949-2e98-47ef-9c63-65ef59e66bf5 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:30:57.461918 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37640949-2e98-47ef-9c63-65ef59e66bf5 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:30:57.462196 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37640949-2e98-47ef-9c63-65ef59e66bf5 None admin] PATH_INFO: `/v3/domains` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:30:57.463536 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37640949-2e98-47ef-9c63-65ef59e66bf5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:30:57.463845 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37640949-2e98-47ef-9c63-65ef59e66bf5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:30:57.465235 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37640949-2e98-47ef-9c63-65ef59e66bf5 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:30:57.479344 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 636/1284] 10.222.0.57 () {56 vars in 1109 bytes} [Fri Jul 1 16:30:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:30:57.485078 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a16d51-26bf-4fdd-8211-f79a1ad7176c None admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:30:57.584775 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a16d51-26bf-4fdd-8211-f79a1ad7176c None admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:30:57.599050 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=100 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:30:57.620273 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e0338e-a5c4-4531-82db-b17c48ba79d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:30:57.621065 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e0338e-a5c4-4531-82db-b17c48ba79d9 None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:30:57.621348 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e0338e-a5c4-4531-82db-b17c48ba79d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:30:57.621622 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e0338e-a5c4-4531-82db-b17c48ba79d9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:30:57.628122 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e0338e-a5c4-4531-82db-b17c48ba79d9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:30:57.629579 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e0338e-a5c4-4531-82db-b17c48ba79d9 None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:30:57.643697 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application [None req-b6e0338e-a5c4-4531-82db-b17c48ba79d9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 01 16:30:57.643697 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 16:30:57.643697 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 16:30:57.643697 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 16:30:57.643697 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 16:30:57.643697 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 16:30:57.643697 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 16:30:57.643697 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 16:30:57.643697 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 16:30:57.643697 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 16:30:57.643697 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 16:30:57.643697 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 16:30:57.643697 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 01 16:30:57.643697 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 01 16:30:57.643697 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 01 16:30:57.644755 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 01 16:30:57.644755 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 16:30:57.644755 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 16:30:57.644755 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 16:30:57.644755 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 16:30:57.644755 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 16:30:57.644755 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 16:30:57.644755 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 16:30:57.644755 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application with Lock( Jul 01 16:30:57.644755 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 16:30:57.644755 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self._enter() Jul 01 16:30:57.644755 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 16:30:57.644755 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 16:30:57.644755 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 16:30:57.644755 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.creator() Jul 01 16:30:57.644755 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 16:30:57.644755 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application created_value = creator( Jul 01 16:30:57.646025 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 01 16:30:57.646025 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 01 16:30:57.646025 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 01 16:30:57.646025 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 01 16:30:57.646025 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 01 16:30:57.646025 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 01 16:30:57.646025 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 01 16:30:57.646025 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: ERROR keystone.server.flask.application Jul 01 16:30:57.646025 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 649/1285] 10.222.0.57 () {56 vars in 1111 bytes} [Fri Jul 1 16:30:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 16:30:57.651241 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37640949-2e98-47ef-9c63-65ef59e66bf5 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:30:57.701787 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:30:57.751525 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37640949-2e98-47ef-9c63-65ef59e66bf5 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:30:57.784743 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705fb9aa-0655-4dcf-9bc8-0c71bee3e760 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:30:57.785397 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705fb9aa-0655-4dcf-9bc8-0c71bee3e760 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:30:57.785679 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705fb9aa-0655-4dcf-9bc8-0c71bee3e760 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:30:57.785954 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705fb9aa-0655-4dcf-9bc8-0c71bee3e760 None admin] PATH_INFO: `/v3/projects` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:30:57.786771 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-705fb9aa-0655-4dcf-9bc8-0c71bee3e760 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:30:57.787085 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-705fb9aa-0655-4dcf-9bc8-0c71bee3e760 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:30:57.788504 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-705fb9aa-0655-4dcf-9bc8-0c71bee3e760 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:30:57.801182 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 637/1286] 10.222.0.57 () {56 vars in 1148 bytes} [Fri Jul 1 16:30:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:30:57.807325 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e0338e-a5c4-4531-82db-b17c48ba79d9 None admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:30:57.906317 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e0338e-a5c4-4531-82db-b17c48ba79d9 None admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:30:57.938266 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063f97bb-7225-4717-9190-3877272488ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:30:57.938988 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063f97bb-7225-4717-9190-3877272488ef None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:30:57.939269 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063f97bb-7225-4717-9190-3877272488ef None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:30:57.939625 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063f97bb-7225-4717-9190-3877272488ef None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:30:57.940443 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-063f97bb-7225-4717-9190-3877272488ef None admin] RBAC: Adding query filter params (role.id=84ee739146934c1ab32c501ff8093644, scope.project.id=e0788e6725a84e57a4174284c1845755, user.id=bbc38af18dec4e9b8143df23af6aad43) {{(pid=80312) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:30:57.940750 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-063f97bb-7225-4717-9190-3877272488ef None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:30:57.942182 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-063f97bb-7225-4717-9190-3877272488ef None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:30:57.948649 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 650/1287] 10.222.0.57 () {56 vars in 1374 bytes} [Fri Jul 1 16:30:57 2022] GET /identity/v3/role_assignments?user.id=bbc38af18dec4e9b8143df23af6aad43&scope.project.id=e0788e6725a84e57a4174284c1845755&role.id=84ee739146934c1ab32c501ff8093644 => generated 608 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:30:59.998010 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b3a113-eebb-46c4-ad2a-3b1425c9156f None None] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:30:59.998593 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b3a113-eebb-46c4-ad2a-3b1425c9156f None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:30:59.998593 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b3a113-eebb-46c4-ad2a-3b1425c9156f None None] PATH_INFO: `/` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:30:59.999986 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 638/1288] 10.222.0.57 () {54 vars in 926 bytes} [Fri Jul 1 16:30:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 16:31:00.005219 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec79210-d006-4d6f-ad3a-e6c749bbe372 None None] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:31:00.005631 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec79210-d006-4d6f-ad3a-e6c749bbe372 None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:31:00.005781 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec79210-d006-4d6f-ad3a-e6c749bbe372 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:31:00.023148 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.auth.core [None req-cec79210-d006-4d6f-ad3a-e6c749bbe372 None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80312) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:31:00.024338 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-cec79210-d006-4d6f-ad3a-e6c749bbe372 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:31:00.054036 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=100 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:31:00.092233 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 651/1289] 10.222.0.57 () {58 vars in 1023 bytes} [Fri Jul 1 16:31:00 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 01 16:31:00.115148 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b3a113-eebb-46c4-ad2a-3b1425c9156f None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:31:00.219871 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b3a113-eebb-46c4-ad2a-3b1425c9156f None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:31:00.230694 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:31:00.254734 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c83787-a847-4a10-a97c-f780319a4fac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:31:00.255531 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c83787-a847-4a10-a97c-f780319a4fac None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:31:00.255819 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c83787-a847-4a10-a97c-f780319a4fac None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:31:00.256099 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c83787-a847-4a10-a97c-f780319a4fac None admin] PATH_INFO: `/v3/services/placement` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:31:00.256754 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c83787-a847-4a10-a97c-f780319a4fac None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:31:00.258113 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c83787-a847-4a10-a97c-f780319a4fac None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:31:00.270254 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application [None req-f0c83787-a847-4a10-a97c-f780319a4fac None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 01 16:31:00.270254 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 16:31:00.270254 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 01 16:31:00.270254 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 01 16:31:00.270254 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 01 16:31:00.270254 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 01 16:31:00.270254 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 01 16:31:00.270254 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 01 16:31:00.270254 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 01 16:31:00.270254 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Jul 01 16:31:00.270254 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 01 16:31:00.270254 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Jul 01 16:31:00.270254 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 16:31:00.270254 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 16:31:00.270254 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 16:31:00.270254 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 16:31:00.270254 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 16:31:00.271512 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 16:31:00.271512 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 16:31:00.271512 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 16:31:00.271512 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 16:31:00.271512 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 16:31:00.271512 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 16:31:00.271512 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 01 16:31:00.271512 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 01 16:31:00.271512 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 01 16:31:00.271512 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 01 16:31:00.271512 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 16:31:00.271512 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 16:31:00.271512 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 16:31:00.271512 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 16:31:00.271512 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 16:31:00.271512 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 16:31:00.272388 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 16:31:00.272388 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application with Lock( Jul 01 16:31:00.272388 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 16:31:00.272388 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._enter() Jul 01 16:31:00.272388 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 16:31:00.272388 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 16:31:00.272388 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 16:31:00.272388 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.creator() Jul 01 16:31:00.272388 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 16:31:00.272388 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application created_value = creator( Jul 01 16:31:00.272388 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 01 16:31:00.272388 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 01 16:31:00.272388 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 01 16:31:00.272388 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Jul 01 16:31:00.272388 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 639/1290] 10.222.0.57 () {56 vars in 1117 bytes} [Fri Jul 1 16:31:00 2022] GET /identity/v3/services/placement => generated 90 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 16:31:00.278193 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec79210-d006-4d6f-ad3a-e6c749bbe372 None None] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:31:00.375697 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec79210-d006-4d6f-ad3a-e6c749bbe372 None None] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:31:00.408857 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df487fa5-ce8a-456f-a427-16844dae291a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:31:00.409516 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df487fa5-ce8a-456f-a427-16844dae291a None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:31:00.409803 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df487fa5-ce8a-456f-a427-16844dae291a None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:31:00.410083 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df487fa5-ce8a-456f-a427-16844dae291a None admin] PATH_INFO: `/v3/services` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:31:00.410825 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df487fa5-ce8a-456f-a427-16844dae291a None admin] RBAC: Adding query filter params (name=placement) {{(pid=80312) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:31:00.411136 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df487fa5-ce8a-456f-a427-16844dae291a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:31:00.412504 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df487fa5-ce8a-456f-a427-16844dae291a None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:31:00.418201 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 652/1291] 10.222.0.57 () {56 vars in 1116 bytes} [Fri Jul 1 16:31:00 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:31:00.423735 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c83787-a847-4a10-a97c-f780319a4fac None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:31:00.522501 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c83787-a847-4a10-a97c-f780319a4fac None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:31:00.560870 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca317d7b-7cc6-443d-bb7f-4af8a0030c2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:31:00.561714 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca317d7b-7cc6-443d-bb7f-4af8a0030c2b None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:31:00.561953 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca317d7b-7cc6-443d-bb7f-4af8a0030c2b None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:31:00.562249 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca317d7b-7cc6-443d-bb7f-4af8a0030c2b None admin] PATH_INFO: `/v3/services` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:31:00.563150 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca317d7b-7cc6-443d-bb7f-4af8a0030c2b None admin] RBAC: Adding query filter params (type=placement) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:31:00.563535 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca317d7b-7cc6-443d-bb7f-4af8a0030c2b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:31:00.565122 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca317d7b-7cc6-443d-bb7f-4af8a0030c2b None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:31:00.570550 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 640/1292] 10.222.0.57 () {56 vars in 1116 bytes} [Fri Jul 1 16:31:00 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:31:02.695272 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7686578-ee57-45f5-93e2-6049b29563c4 None None] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:31:02.695829 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7686578-ee57-45f5-93e2-6049b29563c4 None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:31:02.695829 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7686578-ee57-45f5-93e2-6049b29563c4 None None] PATH_INFO: `/` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:31:02.698129 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 653/1293] 10.222.0.57 () {54 vars in 926 bytes} [Fri Jul 1 16:31:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 16:31:02.703394 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4111f1b2-3cf2-4e14-a5c9-e1169c5ddfc1 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:31:02.703788 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4111f1b2-3cf2-4e14-a5c9-e1169c5ddfc1 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:31:02.703960 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4111f1b2-3cf2-4e14-a5c9-e1169c5ddfc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:31:02.722299 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-4111f1b2-3cf2-4e14-a5c9-e1169c5ddfc1 None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:31:02.723521 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-4111f1b2-3cf2-4e14-a5c9-e1169c5ddfc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:31:02.753157 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:31:02.789393 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 641/1294] 10.222.0.57 () {58 vars in 1023 bytes} [Fri Jul 1 16:31:02 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 01 16:31:02.812529 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7686578-ee57-45f5-93e2-6049b29563c4 None None] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:31:02.844386 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=100 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:31:02.913876 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7686578-ee57-45f5-93e2-6049b29563c4 None None] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:31:02.947628 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167e0440-ab47-40c1-9f1d-37af784f5543 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:31:02.948223 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167e0440-ab47-40c1-9f1d-37af784f5543 None admin] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:31:02.948509 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167e0440-ab47-40c1-9f1d-37af784f5543 None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:31:02.948787 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167e0440-ab47-40c1-9f1d-37af784f5543 None admin] PATH_INFO: `/v3/services` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:31:02.949521 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-167e0440-ab47-40c1-9f1d-37af784f5543 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:31:02.950913 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-167e0440-ab47-40c1-9f1d-37af784f5543 None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:31:02.964253 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 654/1295] 10.222.0.57 () {60 vars in 1141 bytes} [Fri Jul 1 16:31:02 2022] POST /identity/v3/services => generated 246 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 16:31:05.020678 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f206753-ed90-4981-8e0b-e05665f30f28 None None] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:31:05.021204 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f206753-ed90-4981-8e0b-e05665f30f28 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:31:05.021292 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f206753-ed90-4981-8e0b-e05665f30f28 None None] PATH_INFO: `/` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:31:05.022681 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 642/1296] 10.222.0.57 () {54 vars in 926 bytes} [Fri Jul 1 16:31:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 16:31:05.028330 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711b73f9-4a15-4852-9a6e-af4d4ba030b1 None None] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:31:05.028734 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711b73f9-4a15-4852-9a6e-af4d4ba030b1 None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:31:05.028892 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711b73f9-4a15-4852-9a6e-af4d4ba030b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:31:05.045447 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.auth.core [None req-711b73f9-4a15-4852-9a6e-af4d4ba030b1 None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80312) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:31:05.046663 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-711b73f9-4a15-4852-9a6e-af4d4ba030b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:31:05.117876 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 655/1297] 10.222.0.57 () {58 vars in 1023 bytes} [Fri Jul 1 16:31:05 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 01 16:31:05.141885 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f206753-ed90-4981-8e0b-e05665f30f28 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:31:05.240350 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f206753-ed90-4981-8e0b-e05665f30f28 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:31:05.273156 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea01198-97f8-46fe-b407-332bc040a776 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:31:05.273931 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea01198-97f8-46fe-b407-332bc040a776 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:31:05.274211 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea01198-97f8-46fe-b407-332bc040a776 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:31:05.274601 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea01198-97f8-46fe-b407-332bc040a776 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:31:05.275242 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea01198-97f8-46fe-b407-332bc040a776 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:31:05.276591 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea01198-97f8-46fe-b407-332bc040a776 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:31:05.288014 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application [None req-5ea01198-97f8-46fe-b407-332bc040a776 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 01 16:31:05.288014 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 16:31:05.288014 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 01 16:31:05.288014 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 01 16:31:05.288014 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 01 16:31:05.288014 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 01 16:31:05.288014 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 01 16:31:05.288014 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 01 16:31:05.288014 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 01 16:31:05.288014 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Jul 01 16:31:05.288014 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 01 16:31:05.288014 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Jul 01 16:31:05.288014 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 16:31:05.288014 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 16:31:05.288014 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 16:31:05.288014 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 16:31:05.288014 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 16:31:05.289278 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 16:31:05.289278 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 16:31:05.289278 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 16:31:05.289278 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 16:31:05.289278 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 16:31:05.289278 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 16:31:05.289278 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 01 16:31:05.289278 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 01 16:31:05.289278 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 01 16:31:05.289278 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 01 16:31:05.289278 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 16:31:05.289278 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 16:31:05.289278 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 16:31:05.289278 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 16:31:05.289278 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 16:31:05.289278 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 16:31:05.290972 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 16:31:05.290972 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application with Lock( Jul 01 16:31:05.290972 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 16:31:05.290972 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._enter() Jul 01 16:31:05.290972 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 16:31:05.290972 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 16:31:05.290972 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 16:31:05.290972 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.creator() Jul 01 16:31:05.290972 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 16:31:05.290972 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application created_value = creator( Jul 01 16:31:05.290972 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 01 16:31:05.290972 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 01 16:31:05.290972 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 01 16:31:05.290972 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Jul 01 16:31:05.290972 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 643/1298] 10.222.0.57 () {56 vars in 1117 bytes} [Fri Jul 1 16:31:05 2022] GET /identity/v3/services/placement => generated 90 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 16:31:05.295997 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711b73f9-4a15-4852-9a6e-af4d4ba030b1 None None] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:31:05.314736 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:31:05.394419 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711b73f9-4a15-4852-9a6e-af4d4ba030b1 None None] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:31:05.427229 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b9c804-abd4-4fb9-91e5-ad6bda1878fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:31:05.427886 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b9c804-abd4-4fb9-91e5-ad6bda1878fd None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:31:05.428175 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b9c804-abd4-4fb9-91e5-ad6bda1878fd None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:31:05.428456 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b9c804-abd4-4fb9-91e5-ad6bda1878fd None admin] PATH_INFO: `/v3/services` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:31:05.429157 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b9c804-abd4-4fb9-91e5-ad6bda1878fd None admin] RBAC: Adding query filter params (name=placement) {{(pid=80312) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:31:05.429469 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b9c804-abd4-4fb9-91e5-ad6bda1878fd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:31:05.431563 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b9c804-abd4-4fb9-91e5-ad6bda1878fd None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:31:05.436688 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 656/1299] 10.222.0.57 () {56 vars in 1116 bytes} [Fri Jul 1 16:31:05 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:31:05.442604 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea01198-97f8-46fe-b407-332bc040a776 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:31:05.474040 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:31:05.541629 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea01198-97f8-46fe-b407-332bc040a776 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:31:05.576059 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c08b0f-015d-476b-af57-eef560762418 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:31:05.576704 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c08b0f-015d-476b-af57-eef560762418 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:31:05.576988 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c08b0f-015d-476b-af57-eef560762418 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:31:05.577266 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c08b0f-015d-476b-af57-eef560762418 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:31:05.578050 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c08b0f-015d-476b-af57-eef560762418 None admin] RBAC: Adding query filter params (interface=public, service_id=e54c121a707543eda108efb5a3133d25, region_id=RegionOne) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:31:05.578381 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c08b0f-015d-476b-af57-eef560762418 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:31:05.579747 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c08b0f-015d-476b-af57-eef560762418 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:31:05.584757 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 644/1300] 10.222.0.57 () {56 vars in 1251 bytes} [Fri Jul 1 16:31:05 2022] GET /identity/v3/endpoints?service_id=e54c121a707543eda108efb5a3133d25&interface=public®ion_id=RegionOne => generated 194 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:31:05.590996 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b9c804-abd4-4fb9-91e5-ad6bda1878fd None admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:31:05.671734 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=100 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:31:05.690468 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b9c804-abd4-4fb9-91e5-ad6bda1878fd None admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:31:05.724847 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9761a62-0752-4a2d-9b71-b56237636d3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:31:05.725503 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9761a62-0752-4a2d-9b71-b56237636d3b None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:31:05.725783 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9761a62-0752-4a2d-9b71-b56237636d3b None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:31:05.726057 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9761a62-0752-4a2d-9b71-b56237636d3b None admin] PATH_INFO: `/v3/services` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:31:05.726755 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9761a62-0752-4a2d-9b71-b56237636d3b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:31:05.728797 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9761a62-0752-4a2d-9b71-b56237636d3b None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:31:05.735478 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 657/1301] 10.222.0.57 () {56 vars in 1087 bytes} [Fri Jul 1 16:31:05 2022] GET /identity/v3/services => generated 2191 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 16:31:07.792754 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ebf0f5-9c63-4fca-bac5-b8ee720e3369 None None] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:31:07.793284 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ebf0f5-9c63-4fca-bac5-b8ee720e3369 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:31:07.793379 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ebf0f5-9c63-4fca-bac5-b8ee720e3369 None None] PATH_INFO: `/` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:31:07.794731 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 645/1302] 10.222.0.57 () {54 vars in 926 bytes} [Fri Jul 1 16:31:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 16:31:07.799901 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41a2379-ee00-4dfd-a931-919f75966020 None None] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:31:07.800313 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41a2379-ee00-4dfd-a931-919f75966020 None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:31:07.800467 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41a2379-ee00-4dfd-a931-919f75966020 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:31:07.817757 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.auth.core [None req-f41a2379-ee00-4dfd-a931-919f75966020 None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80312) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:31:07.818973 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-f41a2379-ee00-4dfd-a931-919f75966020 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:31:07.887246 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 658/1303] 10.222.0.57 () {58 vars in 1023 bytes} [Fri Jul 1 16:31:07 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 01 16:31:07.913565 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ebf0f5-9c63-4fca-bac5-b8ee720e3369 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:31:08.022224 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:31:08.030451 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ebf0f5-9c63-4fca-bac5-b8ee720e3369 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:31:08.064303 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d7b0be-c9c9-4de8-bad5-e897784e9a6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:31:08.065027 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d7b0be-c9c9-4de8-bad5-e897784e9a6c None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:31:08.065310 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d7b0be-c9c9-4de8-bad5-e897784e9a6c None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:31:08.065581 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d7b0be-c9c9-4de8-bad5-e897784e9a6c None admin] PATH_INFO: `/v3/services/placement` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:31:08.066230 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d7b0be-c9c9-4de8-bad5-e897784e9a6c None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:31:08.067613 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d7b0be-c9c9-4de8-bad5-e897784e9a6c None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:31:08.079863 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application [None req-51d7b0be-c9c9-4de8-bad5-e897784e9a6c None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 01 16:31:08.079863 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 16:31:08.079863 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 01 16:31:08.079863 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 01 16:31:08.079863 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 01 16:31:08.079863 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 01 16:31:08.079863 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 01 16:31:08.079863 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 01 16:31:08.079863 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 01 16:31:08.079863 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Jul 01 16:31:08.079863 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 01 16:31:08.079863 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Jul 01 16:31:08.079863 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 16:31:08.079863 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 16:31:08.079863 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 16:31:08.079863 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 16:31:08.079863 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 16:31:08.081089 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 16:31:08.081089 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 16:31:08.081089 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 16:31:08.081089 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 16:31:08.081089 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 16:31:08.081089 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 16:31:08.081089 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 01 16:31:08.081089 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 01 16:31:08.081089 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 01 16:31:08.081089 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 01 16:31:08.081089 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 16:31:08.081089 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 16:31:08.081089 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 16:31:08.081089 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 16:31:08.081089 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 16:31:08.081089 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 16:31:08.082221 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 16:31:08.082221 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application with Lock( Jul 01 16:31:08.082221 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 16:31:08.082221 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._enter() Jul 01 16:31:08.082221 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 16:31:08.082221 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 16:31:08.082221 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 16:31:08.082221 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.creator() Jul 01 16:31:08.082221 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 16:31:08.082221 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application created_value = creator( Jul 01 16:31:08.082221 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 01 16:31:08.082221 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 01 16:31:08.082221 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 01 16:31:08.082221 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Jul 01 16:31:08.082221 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 646/1304] 10.222.0.57 () {56 vars in 1117 bytes} [Fri Jul 1 16:31:07 2022] GET /identity/v3/services/placement => generated 90 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 16:31:08.088320 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41a2379-ee00-4dfd-a931-919f75966020 None None] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:31:08.163109 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=100 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:31:08.186742 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41a2379-ee00-4dfd-a931-919f75966020 None None] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:31:08.220244 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea486b1-ecb7-42d8-bc21-74d0bf04a431 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:31:08.220902 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea486b1-ecb7-42d8-bc21-74d0bf04a431 None admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:31:08.221189 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea486b1-ecb7-42d8-bc21-74d0bf04a431 None admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:31:08.221469 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea486b1-ecb7-42d8-bc21-74d0bf04a431 None admin] PATH_INFO: `/v3/services` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:31:08.222173 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea486b1-ecb7-42d8-bc21-74d0bf04a431 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80312) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:31:08.222517 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea486b1-ecb7-42d8-bc21-74d0bf04a431 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:31:08.223885 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea486b1-ecb7-42d8-bc21-74d0bf04a431 None admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:31:08.229496 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 659/1305] 10.222.0.57 () {56 vars in 1116 bytes} [Fri Jul 1 16:31:08 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:31:08.235220 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d7b0be-c9c9-4de8-bad5-e897784e9a6c None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:31:08.341317 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d7b0be-c9c9-4de8-bad5-e897784e9a6c None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:31:08.376339 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45d6125-59d9-4b4c-8b39-f8e21124b085 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:31:08.376995 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45d6125-59d9-4b4c-8b39-f8e21124b085 None admin] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:31:08.377283 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45d6125-59d9-4b4c-8b39-f8e21124b085 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:31:08.377566 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45d6125-59d9-4b4c-8b39-f8e21124b085 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:31:08.378302 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c45d6125-59d9-4b4c-8b39-f8e21124b085 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:31:08.379703 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c45d6125-59d9-4b4c-8b39-f8e21124b085 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:31:08.397608 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 647/1306] 10.222.0.57 () {60 vars in 1144 bytes} [Fri Jul 1 16:31:08 2022] POST /identity/v3/endpoints => generated 331 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 16:31:18.226082 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=30 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:31:18.393291 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:32:12.186769 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d786d6-996c-4fc3-8ff3-6ca6d7256f7f None None] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:32:12.187826 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d786d6-996c-4fc3-8ff3-6ca6d7256f7f None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:32:12.188336 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d786d6-996c-4fc3-8ff3-6ca6d7256f7f None None] PATH_INFO: `/` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:32:12.196619 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf21c57e-ca85-42dd-aad6-1d8c3fcfd366 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:32:12.197541 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf21c57e-ca85-42dd-aad6-1d8c3fcfd366 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:32:12.198050 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf21c57e-ca85-42dd-aad6-1d8c3fcfd366 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:32:12.201790 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 660/1307] 10.222.0.57 () {54 vars in 926 bytes} [Fri Jul 1 16:32:12 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 16:32:12.287719 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-cf21c57e-ca85-42dd-aad6-1d8c3fcfd366 None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:32:12.289326 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-cf21c57e-ca85-42dd-aad6-1d8c3fcfd366 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:32:12.418368 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 648/1308] 10.222.0.57 () {58 vars in 1023 bytes} [Fri Jul 1 16:32:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 01 16:32:12.455969 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373d01da-8d71-4142-ad2b-4a27305a78dd None None] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:32:12.456543 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373d01da-8d71-4142-ad2b-4a27305a78dd None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:32:12.456919 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373d01da-8d71-4142-ad2b-4a27305a78dd None None] PATH_INFO: `/` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:32:12.459309 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 661/1309] 10.222.0.57 () {54 vars in 964 bytes} [Fri Jul 1 16:32:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 16:32:12.482102 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4e8c7b-f585-4b02-880e-4b967000d805 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:32:12.483236 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4e8c7b-f585-4b02-880e-4b967000d805 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:32:12.484114 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4e8c7b-f585-4b02-880e-4b967000d805 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:32:12.651557 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-dc4e8c7b-f585-4b02-880e-4b967000d805 None None] MFA Rules not processed for user `4ed200ce29684c5f80354fa9a8638fe2`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:32:12.653100 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-dc4e8c7b-f585-4b02-880e-4b967000d805 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:32:12.675173 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:32:12.791469 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 649/1310] 10.222.0.57 () {58 vars in 1061 bytes} [Fri Jul 1 16:32:12 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 313 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 01 16:32:12.800869 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373d01da-8d71-4142-ad2b-4a27305a78dd None None] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:32:12.802887 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-373d01da-8d71-4142-ad2b-4a27305a78dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:32:12.976904 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373d01da-8d71-4142-ad2b-4a27305a78dd None None] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:32:13.066386 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35152a2-a305-4776-a9f1-01efbc876ab7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ed200ce29684c5f80354fa9a8638fe2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0788e6725a84e57a4174284c1845755', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:32:13.068580 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35152a2-a305-4776-a9f1-01efbc876ab7 service neutron] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:32:13.068972 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35152a2-a305-4776-a9f1-01efbc876ab7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:32:13.069350 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35152a2-a305-4776-a9f1-01efbc876ab7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:32:13.071802 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-e35152a2-a305-4776-a9f1-01efbc876ab7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:32:13.137447 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=100 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:32:13.212334 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e35152a2-a305-4776-a9f1-01efbc876ab7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:32:13.214446 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e35152a2-a305-4776-a9f1-01efbc876ab7 service neutron] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:32:13.238022 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 662/1311] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 1 16:32:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 437 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 16:32:15.678600 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8239c82-5b9c-44dd-9c96-8972489eebfc None None] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:32:15.679682 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8239c82-5b9c-44dd-9c96-8972489eebfc None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:32:15.679682 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8239c82-5b9c-44dd-9c96-8972489eebfc None None] PATH_INFO: `/` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:32:15.680554 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 650/1312] 10.222.0.57 () {54 vars in 926 bytes} [Fri Jul 1 16:32:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 16:32:15.685850 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46eab01e-cd17-4189-ab65-48b6d9bbc21a None None] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:32:15.686273 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46eab01e-cd17-4189-ab65-48b6d9bbc21a None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:32:15.686441 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46eab01e-cd17-4189-ab65-48b6d9bbc21a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:32:15.760114 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.auth.core [None req-46eab01e-cd17-4189-ab65-48b6d9bbc21a None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80312) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:32:15.761350 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-46eab01e-cd17-4189-ab65-48b6d9bbc21a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:32:15.869671 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 663/1313] 10.222.0.57 () {58 vars in 1023 bytes} [Fri Jul 1 16:32:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 16:32:15.909643 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8239c82-5b9c-44dd-9c96-8972489eebfc None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:32:15.911817 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-e8239c82-5b9c-44dd-9c96-8972489eebfc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:32:16.022339 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8239c82-5b9c-44dd-9c96-8972489eebfc None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:32:16.097760 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:32:16.100761 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecafdb92-1e6f-4d34-a4e1-f371253f7e99 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ed200ce29684c5f80354fa9a8638fe2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0788e6725a84e57a4174284c1845755', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:32:16.101541 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecafdb92-1e6f-4d34-a4e1-f371253f7e99 service neutron] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:32:16.102014 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecafdb92-1e6f-4d34-a4e1-f371253f7e99 service neutron] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:32:16.102502 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecafdb92-1e6f-4d34-a4e1-f371253f7e99 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:32:16.104457 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-ecafdb92-1e6f-4d34-a4e1-f371253f7e99 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:32:16.220191 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecafdb92-1e6f-4d34-a4e1-f371253f7e99 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:32:16.221696 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecafdb92-1e6f-4d34-a4e1-f371253f7e99 service neutron] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:32:16.241449 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 651/1314] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 1 16:32:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 16:32:18.621187 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc73b5f9-c9e4-4e5f-9538-016b5fd78ff7 None None] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:32:18.621765 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc73b5f9-c9e4-4e5f-9538-016b5fd78ff7 None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:32:18.621765 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc73b5f9-c9e4-4e5f-9538-016b5fd78ff7 None None] PATH_INFO: `/` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:32:18.623265 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 664/1315] 10.222.0.57 () {54 vars in 926 bytes} [Fri Jul 1 16:32:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 16:32:18.628287 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb27f5a-9cd4-4645-be0b-05c7337e5970 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:32:18.629863 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb27f5a-9cd4-4645-be0b-05c7337e5970 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:32:18.629863 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb27f5a-9cd4-4645-be0b-05c7337e5970 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:32:18.692850 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-edb27f5a-9cd4-4645-be0b-05c7337e5970 None None] MFA Rules not processed for user `dcca9b98066044dca1d7d028b9e83c21`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:32:18.694043 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-edb27f5a-9cd4-4645-be0b-05c7337e5970 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:32:18.745326 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:32:18.789726 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 652/1316] 10.222.0.57 () {58 vars in 1023 bytes} [Fri Jul 1 16:32:18 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 16:32:18.830107 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc73b5f9-c9e4-4e5f-9538-016b5fd78ff7 None None] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:32:18.831743 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-cc73b5f9-c9e4-4e5f-9538-016b5fd78ff7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:32:18.842139 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=100 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:32:18.936407 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc73b5f9-c9e4-4e5f-9538-016b5fd78ff7 None None] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:32:19.009447 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5dc8232-2e3d-4861-80f2-e324a915af76 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ed200ce29684c5f80354fa9a8638fe2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0788e6725a84e57a4174284c1845755', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:32:19.010101 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5dc8232-2e3d-4861-80f2-e324a915af76 service neutron] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:32:19.010416 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5dc8232-2e3d-4861-80f2-e324a915af76 service neutron] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:32:19.010699 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5dc8232-2e3d-4861-80f2-e324a915af76 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:32:19.012359 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-f5dc8232-2e3d-4861-80f2-e324a915af76 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:32:19.074723 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=100 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:32:19.120789 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5dc8232-2e3d-4861-80f2-e324a915af76 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:32:19.122225 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5dc8232-2e3d-4861-80f2-e324a915af76 service neutron] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:32:19.141516 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 665/1317] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 1 16:32:18 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 16:32:21.739872 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d027e7-92b3-49d8-8899-dc7ea790099e None None] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:32:21.740442 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d027e7-92b3-49d8-8899-dc7ea790099e None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:32:21.740442 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d027e7-92b3-49d8-8899-dc7ea790099e None None] PATH_INFO: `/` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:32:21.741824 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 653/1318] 10.222.0.57 () {54 vars in 926 bytes} [Fri Jul 1 16:32:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 16:32:21.747303 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6196cb6b-fb1e-4a12-8846-813888a61750 None None] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:32:21.747753 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6196cb6b-fb1e-4a12-8846-813888a61750 None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:32:21.747848 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6196cb6b-fb1e-4a12-8846-813888a61750 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:32:21.812399 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.auth.core [None req-6196cb6b-fb1e-4a12-8846-813888a61750 None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80312) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:32:21.813595 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-6196cb6b-fb1e-4a12-8846-813888a61750 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:32:21.908799 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 666/1319] 10.222.0.57 () {58 vars in 1023 bytes} [Fri Jul 1 16:32:21 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 16:32:21.963856 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b8dc21-8043-465e-ae8e-c207ab0f8ca9 None None] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:32:21.964149 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b8dc21-8043-465e-ae8e-c207ab0f8ca9 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:32:21.964429 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b8dc21-8043-465e-ae8e-c207ab0f8ca9 None None] PATH_INFO: `/` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:32:21.965844 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 654/1320] 10.222.0.57 () {54 vars in 964 bytes} [Fri Jul 1 16:32:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 16:32:21.971601 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030c3b52-406a-4529-b68a-430d82eff676 None None] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:32:21.971896 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030c3b52-406a-4529-b68a-430d82eff676 None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:32:21.972758 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030c3b52-406a-4529-b68a-430d82eff676 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:32:21.993638 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=100 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:32:22.047496 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.auth.core [None req-030c3b52-406a-4529-b68a-430d82eff676 None None] MFA Rules not processed for user `4ed200ce29684c5f80354fa9a8638fe2`. Rule list: `[]` (Enabled: `True`). {{(pid=80312) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:32:22.048698 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-030c3b52-406a-4529-b68a-430d82eff676 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:32:22.125959 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 667/1321] 10.222.0.57 () {58 vars in 1061 bytes} [Fri Jul 1 16:32:21 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 16:32:22.135729 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b8dc21-8043-465e-ae8e-c207ab0f8ca9 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:32:22.137266 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-76b8dc21-8043-465e-ae8e-c207ab0f8ca9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:32:22.238679 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b8dc21-8043-465e-ae8e-c207ab0f8ca9 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:32:22.310793 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7896a00-6c65-4185-964c-9102c25cb7f5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ed200ce29684c5f80354fa9a8638fe2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0788e6725a84e57a4174284c1845755', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:32:22.311434 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7896a00-6c65-4185-964c-9102c25cb7f5 service neutron] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:32:22.311720 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7896a00-6c65-4185-964c-9102c25cb7f5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:32:22.311997 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7896a00-6c65-4185-964c-9102c25cb7f5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:32:22.313652 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-f7896a00-6c65-4185-964c-9102c25cb7f5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:32:22.321809 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:32:22.422027 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7896a00-6c65-4185-964c-9102c25cb7f5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:32:22.423484 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7896a00-6c65-4185-964c-9102c25cb7f5 service neutron] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:32:22.443175 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 655/1322] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 1 16:32:22 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 16:32:25.626302 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d88331a-6749-42a3-b3ef-ed0854465829 None None] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:32:25.626890 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d88331a-6749-42a3-b3ef-ed0854465829 None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:32:25.626890 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d88331a-6749-42a3-b3ef-ed0854465829 None None] PATH_INFO: `/` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:32:25.628959 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 668/1323] 10.222.0.57 () {54 vars in 926 bytes} [Fri Jul 1 16:32:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 16:32:25.634199 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe02e9f-2d0b-421d-9035-ead3d44f4282 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:32:25.634666 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe02e9f-2d0b-421d-9035-ead3d44f4282 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:32:25.634770 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe02e9f-2d0b-421d-9035-ead3d44f4282 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:32:25.701558 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-ffe02e9f-2d0b-421d-9035-ead3d44f4282 None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:32:25.702843 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-ffe02e9f-2d0b-421d-9035-ead3d44f4282 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:32:25.760942 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:32:25.805008 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 656/1324] 10.222.0.57 () {58 vars in 1023 bytes} [Fri Jul 1 16:32:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 16:32:25.842070 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d88331a-6749-42a3-b3ef-ed0854465829 None None] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:32:25.843693 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-9d88331a-6749-42a3-b3ef-ed0854465829 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:32:25.957184 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d88331a-6749-42a3-b3ef-ed0854465829 None None] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:32:25.970361 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=100 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:32:26.038395 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62cb203-c74f-4fda-9f3d-9ea0cd0c919e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ed200ce29684c5f80354fa9a8638fe2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0788e6725a84e57a4174284c1845755', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:32:26.039059 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62cb203-c74f-4fda-9f3d-9ea0cd0c919e service neutron] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:32:26.039349 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62cb203-c74f-4fda-9f3d-9ea0cd0c919e service neutron] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:32:26.039631 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62cb203-c74f-4fda-9f3d-9ea0cd0c919e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:32:26.041343 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-e62cb203-c74f-4fda-9f3d-9ea0cd0c919e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:32:26.156516 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e62cb203-c74f-4fda-9f3d-9ea0cd0c919e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:32:26.157956 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e62cb203-c74f-4fda-9f3d-9ea0cd0c919e service neutron] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:32:26.177560 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 669/1325] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 1 16:32:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 16:32:28.358995 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644bd65a-9118-4607-8830-d73fe2c9306b None None] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:32:28.359508 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644bd65a-9118-4607-8830-d73fe2c9306b None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:32:28.359793 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644bd65a-9118-4607-8830-d73fe2c9306b None None] PATH_INFO: `/` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:32:28.361215 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 657/1326] 10.222.0.57 () {54 vars in 926 bytes} [Fri Jul 1 16:32:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 16:32:28.367826 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae91ac4a-4c92-4400-9382-2be1cb4d6954 None None] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:32:28.368340 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae91ac4a-4c92-4400-9382-2be1cb4d6954 None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:32:28.368526 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae91ac4a-4c92-4400-9382-2be1cb4d6954 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:32:28.433203 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=100 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:32:28.453122 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.auth.core [None req-ae91ac4a-4c92-4400-9382-2be1cb4d6954 None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80312) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:32:28.454327 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-ae91ac4a-4c92-4400-9382-2be1cb4d6954 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:32:28.576609 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 670/1327] 10.222.0.57 () {58 vars in 1023 bytes} [Fri Jul 1 16:32:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 16:32:28.623808 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644bd65a-9118-4607-8830-d73fe2c9306b None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:32:28.625420 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-644bd65a-9118-4607-8830-d73fe2c9306b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:32:28.734291 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644bd65a-9118-4607-8830-d73fe2c9306b None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:32:28.809705 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b002b1d4-7916-40f0-b243-84a8ef41ba26 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ed200ce29684c5f80354fa9a8638fe2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0788e6725a84e57a4174284c1845755', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:32:28.810402 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b002b1d4-7916-40f0-b243-84a8ef41ba26 service neutron] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:32:28.810692 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b002b1d4-7916-40f0-b243-84a8ef41ba26 service neutron] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:32:28.810973 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b002b1d4-7916-40f0-b243-84a8ef41ba26 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:32:28.812729 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-b002b1d4-7916-40f0-b243-84a8ef41ba26 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:32:28.840302 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:32:28.948107 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b002b1d4-7916-40f0-b243-84a8ef41ba26 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:32:28.949551 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b002b1d4-7916-40f0-b243-84a8ef41ba26 service neutron] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:32:28.968865 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 658/1328] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 1 16:32:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 16:32:31.099365 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5758e574-35e2-4c6f-a40f-4a448faf00fd None None] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:32:31.099885 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5758e574-35e2-4c6f-a40f-4a448faf00fd None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:32:31.099885 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5758e574-35e2-4c6f-a40f-4a448faf00fd None None] PATH_INFO: `/` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:32:31.101237 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 671/1329] 10.222.0.57 () {54 vars in 926 bytes} [Fri Jul 1 16:32:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 16:32:31.106592 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d92f00-5768-40f6-b459-4731d211bf2b None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:32:31.107045 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d92f00-5768-40f6-b459-4731d211bf2b None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:32:31.107266 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d92f00-5768-40f6-b459-4731d211bf2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:32:31.176630 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-e1d92f00-5768-40f6-b459-4731d211bf2b None None] MFA Rules not processed for user `dcca9b98066044dca1d7d028b9e83c21`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:32:31.177822 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-e1d92f00-5768-40f6-b459-4731d211bf2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:32:31.246377 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:32:31.281693 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 659/1330] 10.222.0.57 () {58 vars in 1023 bytes} [Fri Jul 1 16:32:31 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 16:32:31.321505 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5758e574-35e2-4c6f-a40f-4a448faf00fd None None] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:32:31.323072 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-5758e574-35e2-4c6f-a40f-4a448faf00fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:32:31.425893 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5758e574-35e2-4c6f-a40f-4a448faf00fd None None] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:32:31.447834 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=100 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:32:31.503543 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3689f69-cf05-47a1-8cb4-7d4db1f68800 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ed200ce29684c5f80354fa9a8638fe2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0788e6725a84e57a4174284c1845755', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:32:31.504206 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3689f69-cf05-47a1-8cb4-7d4db1f68800 service neutron] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:32:31.504490 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3689f69-cf05-47a1-8cb4-7d4db1f68800 service neutron] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:32:31.504770 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3689f69-cf05-47a1-8cb4-7d4db1f68800 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:32:31.506485 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-b3689f69-cf05-47a1-8cb4-7d4db1f68800 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:32:31.618519 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3689f69-cf05-47a1-8cb4-7d4db1f68800 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:32:31.619957 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3689f69-cf05-47a1-8cb4-7d4db1f68800 service neutron] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:32:31.638245 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 672/1331] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 1 16:32:31 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 16:32:33.975969 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431786f7-804f-4499-b116-44f5c5396168 None None] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:32:33.976529 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431786f7-804f-4499-b116-44f5c5396168 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:32:33.976529 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431786f7-804f-4499-b116-44f5c5396168 None None] PATH_INFO: `/` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:32:33.977904 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 660/1332] 10.222.0.57 () {54 vars in 926 bytes} [Fri Jul 1 16:32:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 16:32:33.983245 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8341db0-eb86-438d-a227-59bfc07e575f None None] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:32:33.983633 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8341db0-eb86-438d-a227-59bfc07e575f None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:32:33.983944 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8341db0-eb86-438d-a227-59bfc07e575f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:32:34.046192 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=100 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:32:34.049112 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.auth.core [None req-a8341db0-eb86-438d-a227-59bfc07e575f None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80312) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:32:34.051444 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-a8341db0-eb86-438d-a227-59bfc07e575f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:32:34.146785 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 673/1333] 10.222.0.57 () {58 vars in 1023 bytes} [Fri Jul 1 16:32:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 16:32:34.182548 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431786f7-804f-4499-b116-44f5c5396168 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:32:34.184116 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-431786f7-804f-4499-b116-44f5c5396168 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:32:34.293320 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431786f7-804f-4499-b116-44f5c5396168 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:32:34.370820 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b78ea3-8408-4ecb-9f73-cb577b911867 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ed200ce29684c5f80354fa9a8638fe2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0788e6725a84e57a4174284c1845755', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:32:34.371678 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b78ea3-8408-4ecb-9f73-cb577b911867 service neutron] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:32:34.372047 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b78ea3-8408-4ecb-9f73-cb577b911867 service neutron] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:32:34.372415 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b78ea3-8408-4ecb-9f73-cb577b911867 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:32:34.374185 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-79b78ea3-8408-4ecb-9f73-cb577b911867 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:32:34.403570 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:32:34.487504 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b78ea3-8408-4ecb-9f73-cb577b911867 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:32:34.488939 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b78ea3-8408-4ecb-9f73-cb577b911867 service neutron] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:32:34.507592 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 661/1334] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 1 16:32:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 16:32:36.618520 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef4ba59-12f0-44a3-9a71-bddce420099c None None] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:32:36.619097 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef4ba59-12f0-44a3-9a71-bddce420099c None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:32:36.619097 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef4ba59-12f0-44a3-9a71-bddce420099c None None] PATH_INFO: `/` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:32:36.620471 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 674/1335] 10.222.0.57 () {54 vars in 926 bytes} [Fri Jul 1 16:32:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 16:32:36.625743 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8f9ce4-f98a-4657-95ba-cdc8a7c4d583 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:32:36.626169 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8f9ce4-f98a-4657-95ba-cdc8a7c4d583 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:32:36.626310 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8f9ce4-f98a-4657-95ba-cdc8a7c4d583 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:32:36.694203 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-0c8f9ce4-f98a-4657-95ba-cdc8a7c4d583 None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:32:36.695712 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-0c8f9ce4-f98a-4657-95ba-cdc8a7c4d583 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:32:36.769452 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:32:36.793674 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 662/1336] 10.222.0.57 () {58 vars in 1023 bytes} [Fri Jul 1 16:32:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 16:32:36.834185 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef4ba59-12f0-44a3-9a71-bddce420099c None None] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:32:36.835780 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-0ef4ba59-12f0-44a3-9a71-bddce420099c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:32:36.939808 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef4ba59-12f0-44a3-9a71-bddce420099c None None] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:32:36.967923 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=100 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:32:37.012988 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3a2635-7dcd-4b0f-8a13-b38d5bd401e8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ed200ce29684c5f80354fa9a8638fe2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0788e6725a84e57a4174284c1845755', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:32:37.013631 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3a2635-7dcd-4b0f-8a13-b38d5bd401e8 service neutron] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:32:37.013914 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3a2635-7dcd-4b0f-8a13-b38d5bd401e8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:32:37.014189 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3a2635-7dcd-4b0f-8a13-b38d5bd401e8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:32:37.015847 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-6a3a2635-7dcd-4b0f-8a13-b38d5bd401e8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:32:37.122876 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a3a2635-7dcd-4b0f-8a13-b38d5bd401e8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:32:37.124295 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a3a2635-7dcd-4b0f-8a13-b38d5bd401e8 service neutron] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:32:37.142104 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 675/1337] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 1 16:32:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 16:32:39.552868 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4decdea4-256e-4dc0-826d-775cca71e72d None None] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:32:39.553451 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4decdea4-256e-4dc0-826d-775cca71e72d None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:32:39.553451 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4decdea4-256e-4dc0-826d-775cca71e72d None None] PATH_INFO: `/` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:32:39.554859 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 663/1338] 10.222.0.57 () {54 vars in 926 bytes} [Fri Jul 1 16:32:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 16:32:39.560177 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3c3778-2993-45de-bf9e-47b6439cddb2 None None] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:32:39.560608 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3c3778-2993-45de-bf9e-47b6439cddb2 None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:32:39.560767 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3c3778-2993-45de-bf9e-47b6439cddb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:32:39.625917 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.auth.core [None req-5e3c3778-2993-45de-bf9e-47b6439cddb2 None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80312) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:32:39.627224 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-5e3c3778-2993-45de-bf9e-47b6439cddb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:32:39.635660 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=100 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:32:39.730845 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 676/1339] 10.222.0.57 () {58 vars in 1023 bytes} [Fri Jul 1 16:32:39 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 16:32:39.770561 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4decdea4-256e-4dc0-826d-775cca71e72d None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:32:39.772103 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-4decdea4-256e-4dc0-826d-775cca71e72d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:32:39.881086 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4decdea4-256e-4dc0-826d-775cca71e72d None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:32:39.952493 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9ddc11-cfa4-45d1-a323-c410d881bb2d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ed200ce29684c5f80354fa9a8638fe2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0788e6725a84e57a4174284c1845755', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:32:39.953160 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9ddc11-cfa4-45d1-a323-c410d881bb2d service neutron] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:32:39.953441 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9ddc11-cfa4-45d1-a323-c410d881bb2d service neutron] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:32:39.953714 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9ddc11-cfa4-45d1-a323-c410d881bb2d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:32:39.955408 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-9d9ddc11-cfa4-45d1-a323-c410d881bb2d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:32:39.990674 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:32:40.064141 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d9ddc11-cfa4-45d1-a323-c410d881bb2d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:32:40.065796 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d9ddc11-cfa4-45d1-a323-c410d881bb2d service neutron] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:32:40.084816 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 664/1340] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 1 16:32:39 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 01 16:32:44.263200 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ae565c-6761-49ff-b532-436541d93f4a None None] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:32:44.263882 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ae565c-6761-49ff-b532-436541d93f4a None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:32:44.263882 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ae565c-6761-49ff-b532-436541d93f4a None None] PATH_INFO: `/` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:32:44.269734 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 677/1341] 10.222.0.57 () {54 vars in 926 bytes} [Fri Jul 1 16:32:44 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 16:32:44.272149 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b3d371-1eca-4d72-923c-716d70b626de None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:32:44.272556 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b3d371-1eca-4d72-923c-716d70b626de None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:32:44.273400 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b3d371-1eca-4d72-923c-716d70b626de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:32:44.379587 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-29b3d371-1eca-4d72-923c-716d70b626de None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:32:44.380790 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-29b3d371-1eca-4d72-923c-716d70b626de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:32:44.489731 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:32:44.505480 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 665/1342] 10.222.0.57 () {58 vars in 1023 bytes} [Fri Jul 1 16:32:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 234 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 16:32:44.546696 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ae565c-6761-49ff-b532-436541d93f4a None None] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:32:44.551097 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-26ae565c-6761-49ff-b532-436541d93f4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:32:44.661680 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ae565c-6761-49ff-b532-436541d93f4a None None] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:32:44.665496 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2e49f6-e6b5-4fe8-a4ee-add20939d665 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ed200ce29684c5f80354fa9a8638fe2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0788e6725a84e57a4174284c1845755', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:32:44.666168 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2e49f6-e6b5-4fe8-a4ee-add20939d665 service neutron] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:32:44.666476 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2e49f6-e6b5-4fe8-a4ee-add20939d665 service neutron] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:32:44.666758 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2e49f6-e6b5-4fe8-a4ee-add20939d665 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:32:44.678084 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f2e49f6-e6b5-4fe8-a4ee-add20939d665 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:32:44.679568 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f2e49f6-e6b5-4fe8-a4ee-add20939d665 service neutron] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:32:44.690860 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 678/1343] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 1 16:32:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 16:32:47.479930 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce1031e-0366-47d2-92a6-3e17d3e3ed05 None None] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:32:47.480543 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce1031e-0366-47d2-92a6-3e17d3e3ed05 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:32:47.480543 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce1031e-0366-47d2-92a6-3e17d3e3ed05 None None] PATH_INFO: `/` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:32:47.482009 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 666/1344] 10.222.0.57 () {54 vars in 926 bytes} [Fri Jul 1 16:32:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 16:32:47.488264 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c63b898-0208-4dfa-a3cc-4dddf43a1243 None None] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:32:47.488792 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c63b898-0208-4dfa-a3cc-4dddf43a1243 None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:32:47.488897 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c63b898-0208-4dfa-a3cc-4dddf43a1243 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:32:47.516188 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.auth.core [None req-0c63b898-0208-4dfa-a3cc-4dddf43a1243 None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80312) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:32:47.517421 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-0c63b898-0208-4dfa-a3cc-4dddf43a1243 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:32:47.546291 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=100 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:32:47.608839 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 679/1345] 10.222.0.57 () {58 vars in 1023 bytes} [Fri Jul 1 16:32:47 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 01 16:32:47.648550 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce1031e-0366-47d2-92a6-3e17d3e3ed05 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:32:47.659520 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce1031e-0366-47d2-92a6-3e17d3e3ed05 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:32:47.663097 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d010eb7-37f5-4fea-8fbf-894840935083 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ed200ce29684c5f80354fa9a8638fe2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0788e6725a84e57a4174284c1845755', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:32:47.663749 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d010eb7-37f5-4fea-8fbf-894840935083 service neutron] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:32:47.664029 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d010eb7-37f5-4fea-8fbf-894840935083 service neutron] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:32:47.664302 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d010eb7-37f5-4fea-8fbf-894840935083 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:32:47.674622 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d010eb7-37f5-4fea-8fbf-894840935083 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:32:47.676374 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d010eb7-37f5-4fea-8fbf-894840935083 service neutron] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:32:47.686602 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 667/1346] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 1 16:32:47 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 16:32:52.032369 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749db520-be13-45ae-9e4f-5e423d182cad None None] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:32:52.033318 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749db520-be13-45ae-9e4f-5e423d182cad None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:32:52.033476 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749db520-be13-45ae-9e4f-5e423d182cad None None] PATH_INFO: `/` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:32:52.034884 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 680/1347] 10.222.0.57 () {54 vars in 926 bytes} [Fri Jul 1 16:32:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 16:32:52.040272 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861c9bcb-9ecf-455d-ac20-4e8628fed01d None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:32:52.040729 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861c9bcb-9ecf-455d-ac20-4e8628fed01d None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:32:52.040827 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861c9bcb-9ecf-455d-ac20-4e8628fed01d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:32:52.059266 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-861c9bcb-9ecf-455d-ac20-4e8628fed01d None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:32:52.060521 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-861c9bcb-9ecf-455d-ac20-4e8628fed01d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:32:52.068571 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 668/1348] 10.222.0.57 () {58 vars in 1023 bytes} [Fri Jul 1 16:32:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 16:32:52.107670 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749db520-be13-45ae-9e4f-5e423d182cad None None] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:32:52.109393 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-749db520-be13-45ae-9e4f-5e423d182cad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:32:52.122797 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749db520-be13-45ae-9e4f-5e423d182cad None None] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:32:52.126804 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289d0e45-9c0e-4138-9c3d-769710f60341 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ed200ce29684c5f80354fa9a8638fe2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0788e6725a84e57a4174284c1845755', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:32:52.127571 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289d0e45-9c0e-4138-9c3d-769710f60341 service neutron] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:32:52.127740 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289d0e45-9c0e-4138-9c3d-769710f60341 service neutron] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:32:52.128022 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289d0e45-9c0e-4138-9c3d-769710f60341 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:32:52.138059 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-289d0e45-9c0e-4138-9c3d-769710f60341 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:32:52.139559 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-289d0e45-9c0e-4138-9c3d-769710f60341 service neutron] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:32:52.150126 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 681/1349] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 1 16:32:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 16:32:57.233445 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a143bd-5f47-4ca2-9344-645994731e81 None None] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:32:57.233957 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a143bd-5f47-4ca2-9344-645994731e81 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:32:57.234245 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a143bd-5f47-4ca2-9344-645994731e81 None None] PATH_INFO: `/` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:32:57.236655 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 669/1350] 10.222.0.57 () {54 vars in 921 bytes} [Fri Jul 1 16:32:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 16:32:57.245953 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ee33b5-5d3f-4b95-8597-d97d06035809 None None] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:32:57.245953 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ee33b5-5d3f-4b95-8597-d97d06035809 None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:32:57.245953 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ee33b5-5d3f-4b95-8597-d97d06035809 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:32:57.416130 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.auth.core [None req-b6ee33b5-5d3f-4b95-8597-d97d06035809 None None] MFA Rules not processed for user `bbc38af18dec4e9b8143df23af6aad43`. Rule list: `[]` (Enabled: `True`). {{(pid=80312) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:32:57.417261 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-b6ee33b5-5d3f-4b95-8597-d97d06035809 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:32:57.518066 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 682/1351] 10.222.0.57 () {58 vars in 1018 bytes} [Fri Jul 1 16:32:57 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 276 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 16:33:01.671123 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6789b9-b899-48ab-8ec5-61552a9e90e0 None None] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:33:01.671654 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6789b9-b899-48ab-8ec5-61552a9e90e0 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:33:01.672201 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6789b9-b899-48ab-8ec5-61552a9e90e0 None None] PATH_INFO: `/` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:33:01.676372 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 670/1352] 10.222.0.57 () {54 vars in 921 bytes} [Fri Jul 1 16:33:01 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 16:33:01.681379 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100ea3c2-9b33-4f5d-95fe-20bb55cd0e68 None None] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:33:01.682519 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100ea3c2-9b33-4f5d-95fe-20bb55cd0e68 None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:33:01.683031 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100ea3c2-9b33-4f5d-95fe-20bb55cd0e68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:33:01.702499 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.auth.core [None req-100ea3c2-9b33-4f5d-95fe-20bb55cd0e68 None None] MFA Rules not processed for user `bbc38af18dec4e9b8143df23af6aad43`. Rule list: `[]` (Enabled: `True`). {{(pid=80312) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:33:01.704277 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-100ea3c2-9b33-4f5d-95fe-20bb55cd0e68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:33:01.713533 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 683/1353] 10.222.0.57 () {58 vars in 1018 bytes} [Fri Jul 1 16:33:01 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 16:33:02.054256 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=13 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:33:03.288297 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0bb54c9-616e-47c1-8cb1-935147a10c19 None None] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:33:03.288929 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0bb54c9-616e-47c1-8cb1-935147a10c19 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:33:03.289090 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0bb54c9-616e-47c1-8cb1-935147a10c19 None None] PATH_INFO: `/` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:33:03.290523 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 671/1354] 10.222.0.57 () {54 vars in 921 bytes} [Fri Jul 1 16:33:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 16:33:03.295711 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25192ff5-f089-4c07-8da4-2f73bfda2ec3 None None] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:33:03.296097 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25192ff5-f089-4c07-8da4-2f73bfda2ec3 None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:33:03.296385 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25192ff5-f089-4c07-8da4-2f73bfda2ec3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:33:03.315456 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.auth.core [None req-25192ff5-f089-4c07-8da4-2f73bfda2ec3 None None] MFA Rules not processed for user `bbc38af18dec4e9b8143df23af6aad43`. Rule list: `[]` (Enabled: `True`). {{(pid=80312) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:33:03.316733 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-25192ff5-f089-4c07-8da4-2f73bfda2ec3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:33:03.325553 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 684/1355] 10.222.0.57 () {58 vars in 1018 bytes} [Fri Jul 1 16:33:03 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 01 16:33:07.065066 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1a803f-7d93-4db2-bb66-74dbd0cc7c6b None None] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:33:07.066000 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1a803f-7d93-4db2-bb66-74dbd0cc7c6b None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:33:07.067527 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1a803f-7d93-4db2-bb66-74dbd0cc7c6b None None] PATH_INFO: `/` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:33:07.069791 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 672/1356] 10.222.0.57 () {54 vars in 926 bytes} [Fri Jul 1 16:33:07 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 16:33:07.075820 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30486f5c-7769-4d23-9619-d2b29348a5f9 None None] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:33:07.077160 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30486f5c-7769-4d23-9619-d2b29348a5f9 None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:33:07.077793 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30486f5c-7769-4d23-9619-d2b29348a5f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:33:07.102016 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.auth.core [None req-30486f5c-7769-4d23-9619-d2b29348a5f9 None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80312) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:33:07.103821 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-30486f5c-7769-4d23-9619-d2b29348a5f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:33:07.116804 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 685/1357] 10.222.0.57 () {58 vars in 1023 bytes} [Fri Jul 1 16:33:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 16:33:07.181519 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f4a63a-59c0-4b4c-8f5d-ef9cc04bc752 None None] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:33:07.181815 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f4a63a-59c0-4b4c-8f5d-ef9cc04bc752 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:33:07.182101 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f4a63a-59c0-4b4c-8f5d-ef9cc04bc752 None None] PATH_INFO: `/` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:33:07.183799 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 673/1358] 10.222.0.57 () {54 vars in 961 bytes} [Fri Jul 1 16:33:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 16:33:07.194330 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce422e2-bd49-4d20-9349-be5ad2cba3d8 None None] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:33:07.195059 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce422e2-bd49-4d20-9349-be5ad2cba3d8 None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:33:07.195557 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce422e2-bd49-4d20-9349-be5ad2cba3d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:33:07.246191 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=100 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:33:07.271845 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.auth.core [None req-bce422e2-bd49-4d20-9349-be5ad2cba3d8 None None] MFA Rules not processed for user `7a50f745b31941b7a7265c0fc852f48e`. Rule list: `[]` (Enabled: `True`). {{(pid=80312) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:33:07.273455 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-bce422e2-bd49-4d20-9349-be5ad2cba3d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:33:07.369167 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 686/1359] 10.222.0.57 () {58 vars in 1058 bytes} [Fri Jul 1 16:33:07 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 01 16:33:07.377328 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f4a63a-59c0-4b4c-8f5d-ef9cc04bc752 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:33:07.393511 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f4a63a-59c0-4b4c-8f5d-ef9cc04bc752 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:33:07.399437 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a347d3-ed88-4cc2-91aa-ff40aea30687 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a50f745b31941b7a7265c0fc852f48e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0788e6725a84e57a4174284c1845755', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:33:07.400337 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a347d3-ed88-4cc2-91aa-ff40aea30687 service nova] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:33:07.400887 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a347d3-ed88-4cc2-91aa-ff40aea30687 service nova] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:33:07.401374 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a347d3-ed88-4cc2-91aa-ff40aea30687 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:33:07.410682 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a347d3-ed88-4cc2-91aa-ff40aea30687 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:33:07.412574 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a347d3-ed88-4cc2-91aa-ff40aea30687 service nova] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:33:07.423826 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 674/1360] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 1 16:33:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 16:33:09.721837 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cd6fdb-e7d7-4901-be70-e739594c7ff8 None None] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:33:09.722521 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cd6fdb-e7d7-4901-be70-e739594c7ff8 None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:33:09.722521 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cd6fdb-e7d7-4901-be70-e739594c7ff8 None None] PATH_INFO: `/` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:33:09.725593 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 687/1361] 10.222.0.57 () {54 vars in 926 bytes} [Fri Jul 1 16:33:09 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 16:33:09.729304 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8daf84-9735-4450-8e8a-9b82df2e2e80 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:33:09.730139 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8daf84-9735-4450-8e8a-9b82df2e2e80 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:33:09.730689 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8daf84-9735-4450-8e8a-9b82df2e2e80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:33:09.749905 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-0a8daf84-9735-4450-8e8a-9b82df2e2e80 None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:33:09.751235 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-0a8daf84-9735-4450-8e8a-9b82df2e2e80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:33:09.759030 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 675/1362] 10.222.0.57 () {58 vars in 1023 bytes} [Fri Jul 1 16:33:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 16:33:09.804535 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69cd6fdb-e7d7-4901-be70-e739594c7ff8 None None] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:33:09.819330 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69cd6fdb-e7d7-4901-be70-e739594c7ff8 None None] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:33:09.824706 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08bfe020-cee4-4188-899b-06513a40bcf6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a50f745b31941b7a7265c0fc852f48e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0788e6725a84e57a4174284c1845755', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:33:09.825348 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bfe020-cee4-4188-899b-06513a40bcf6 service nova] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:33:09.825635 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bfe020-cee4-4188-899b-06513a40bcf6 service nova] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:33:09.825915 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bfe020-cee4-4188-899b-06513a40bcf6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:33:09.837927 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08bfe020-cee4-4188-899b-06513a40bcf6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:33:09.839789 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08bfe020-cee4-4188-899b-06513a40bcf6 service nova] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:33:09.853406 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 688/1363] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 1 16:33:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 16:33:12.323834 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ec3913-b8cc-49f0-81d5-f03eef52ee55 None None] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:33:12.324342 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ec3913-b8cc-49f0-81d5-f03eef52ee55 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:33:12.324469 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ec3913-b8cc-49f0-81d5-f03eef52ee55 None None] PATH_INFO: `/` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:33:12.325758 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 676/1364] 10.222.0.57 () {54 vars in 926 bytes} [Fri Jul 1 16:33:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 16:33:12.331697 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8102405-9be7-49b8-896e-18507b16a1fc None None] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:33:12.332141 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8102405-9be7-49b8-896e-18507b16a1fc None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:33:12.332239 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8102405-9be7-49b8-896e-18507b16a1fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:33:12.350741 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.auth.core [None req-b8102405-9be7-49b8-896e-18507b16a1fc None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80312) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:33:12.351920 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-b8102405-9be7-49b8-896e-18507b16a1fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:33:12.360259 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 689/1365] 10.222.0.57 () {58 vars in 1023 bytes} [Fri Jul 1 16:33:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 16:33:12.406080 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26cb6349-8f54-424a-a671-e2388e609699 None None] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:33:12.407625 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26cb6349-8f54-424a-a671-e2388e609699 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:33:12.407901 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26cb6349-8f54-424a-a671-e2388e609699 None None] PATH_INFO: `/` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:33:12.409437 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 677/1366] 10.222.0.57 () {54 vars in 961 bytes} [Fri Jul 1 16:33:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 16:33:12.414948 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67add8e-b420-4cef-b7ea-1dcacce33138 None None] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:33:12.415316 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67add8e-b420-4cef-b7ea-1dcacce33138 None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:33:12.415596 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67add8e-b420-4cef-b7ea-1dcacce33138 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:33:12.433822 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.auth.core [None req-b67add8e-b420-4cef-b7ea-1dcacce33138 None None] MFA Rules not processed for user `7a50f745b31941b7a7265c0fc852f48e`. Rule list: `[]` (Enabled: `True`). {{(pid=80312) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:33:12.435041 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-b67add8e-b420-4cef-b7ea-1dcacce33138 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:33:12.444776 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 690/1367] 10.222.0.57 () {58 vars in 1058 bytes} [Fri Jul 1 16:33:12 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 16:33:12.452344 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26cb6349-8f54-424a-a671-e2388e609699 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:33:12.466984 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26cb6349-8f54-424a-a671-e2388e609699 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:33:12.472984 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7035e69c-34f2-451c-a41d-9e7f98910acb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a50f745b31941b7a7265c0fc852f48e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0788e6725a84e57a4174284c1845755', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:33:12.473709 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7035e69c-34f2-451c-a41d-9e7f98910acb service nova] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:33:12.474057 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7035e69c-34f2-451c-a41d-9e7f98910acb service nova] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:33:12.474430 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7035e69c-34f2-451c-a41d-9e7f98910acb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:33:12.482768 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7035e69c-34f2-451c-a41d-9e7f98910acb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:33:12.484441 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7035e69c-34f2-451c-a41d-9e7f98910acb service nova] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:33:12.494220 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 678/1368] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 1 16:33:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 16:33:14.651564 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578e9fb6-5458-4b2b-9069-d75125cb4339 None None] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:33:14.652113 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578e9fb6-5458-4b2b-9069-d75125cb4339 None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:33:14.652113 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578e9fb6-5458-4b2b-9069-d75125cb4339 None None] PATH_INFO: `/` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:33:14.653489 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 691/1369] 10.222.0.57 () {54 vars in 926 bytes} [Fri Jul 1 16:33:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 16:33:14.660716 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdd2c8f-4e10-42ad-a6f6-902b1d3ba816 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:33:14.661415 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdd2c8f-4e10-42ad-a6f6-902b1d3ba816 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:33:14.661865 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdd2c8f-4e10-42ad-a6f6-902b1d3ba816 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:33:14.682843 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-dfdd2c8f-4e10-42ad-a6f6-902b1d3ba816 None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:33:14.684051 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-dfdd2c8f-4e10-42ad-a6f6-902b1d3ba816 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:33:14.692953 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 679/1370] 10.222.0.57 () {58 vars in 1023 bytes} [Fri Jul 1 16:33:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 16:33:14.739177 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578e9fb6-5458-4b2b-9069-d75125cb4339 None None] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:33:14.754496 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578e9fb6-5458-4b2b-9069-d75125cb4339 None None] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:33:14.760359 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1eb92fa-4d5c-4405-9376-86b8d24b498d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a50f745b31941b7a7265c0fc852f48e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0788e6725a84e57a4174284c1845755', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:33:14.761062 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1eb92fa-4d5c-4405-9376-86b8d24b498d service nova] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:33:14.761409 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1eb92fa-4d5c-4405-9376-86b8d24b498d service nova] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:33:14.761754 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1eb92fa-4d5c-4405-9376-86b8d24b498d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:33:14.770658 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1eb92fa-4d5c-4405-9376-86b8d24b498d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:33:14.772303 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1eb92fa-4d5c-4405-9376-86b8d24b498d service nova] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:33:14.782956 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 692/1371] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 1 16:33:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 16:33:15.414906 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db4bd2a-9621-4dbd-9072-c697b10f6b4b None None] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:33:15.415184 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db4bd2a-9621-4dbd-9072-c697b10f6b4b None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:33:15.415461 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db4bd2a-9621-4dbd-9072-c697b10f6b4b None None] PATH_INFO: `/` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:33:15.417050 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 680/1372] 10.222.0.57 () {54 vars in 926 bytes} [Fri Jul 1 16:33:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 16:33:15.423033 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883c3c57-1fda-445e-8067-e7fe1a69bf56 None None] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:33:15.423325 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883c3c57-1fda-445e-8067-e7fe1a69bf56 None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:33:15.423602 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883c3c57-1fda-445e-8067-e7fe1a69bf56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:33:15.441743 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.auth.core [None req-883c3c57-1fda-445e-8067-e7fe1a69bf56 None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80312) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:33:15.442994 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-883c3c57-1fda-445e-8067-e7fe1a69bf56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:33:15.454377 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 693/1373] 10.222.0.57 () {58 vars in 1023 bytes} [Fri Jul 1 16:33:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 16:33:15.810110 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4765677-5b17-4d1a-8540-287e8a738929 None None] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:33:15.810683 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4765677-5b17-4d1a-8540-287e8a738929 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:33:15.810822 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4765677-5b17-4d1a-8540-287e8a738929 None None] PATH_INFO: `/` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:33:15.814384 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 681/1374] 10.222.0.57 () {54 vars in 963 bytes} [Fri Jul 1 16:33:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 16:33:15.817588 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52139bb-b91a-41e9-b35f-296c6946af22 None None] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:33:15.818290 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52139bb-b91a-41e9-b35f-296c6946af22 None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:33:15.818663 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52139bb-b91a-41e9-b35f-296c6946af22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:33:15.878138 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.auth.core [None req-d52139bb-b91a-41e9-b35f-296c6946af22 None None] MFA Rules not processed for user `dbddf971a2354bbdbb9da1d63f3abf88`. Rule list: `[]` (Enabled: `True`). {{(pid=80312) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:33:15.879519 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-d52139bb-b91a-41e9-b35f-296c6946af22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:33:15.947208 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 694/1375] 10.222.0.57 () {58 vars in 1060 bytes} [Fri Jul 1 16:33:15 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 16:33:15.955044 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4765677-5b17-4d1a-8540-287e8a738929 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:33:15.966241 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4765677-5b17-4d1a-8540-287e8a738929 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:33:15.971694 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4967cff5-1f4e-4413-9316-0ecc99d14a2b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbddf971a2354bbdbb9da1d63f3abf88', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0788e6725a84e57a4174284c1845755', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:33:15.972508 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4967cff5-1f4e-4413-9316-0ecc99d14a2b service cinder] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:33:15.972897 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4967cff5-1f4e-4413-9316-0ecc99d14a2b service cinder] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:33:15.973297 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4967cff5-1f4e-4413-9316-0ecc99d14a2b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:33:15.983757 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4967cff5-1f4e-4413-9316-0ecc99d14a2b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:33:15.986547 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4967cff5-1f4e-4413-9316-0ecc99d14a2b service cinder] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:33:15.997519 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 682/1376] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 1 16:33:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 16:33:16.919023 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ae6b02-9baa-45ba-9dae-dc5026be5095 None None] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:33:16.919657 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ae6b02-9baa-45ba-9dae-dc5026be5095 None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:33:16.919657 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ae6b02-9baa-45ba-9dae-dc5026be5095 None None] PATH_INFO: `/` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:33:16.921148 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 695/1377] 10.222.0.57 () {54 vars in 926 bytes} [Fri Jul 1 16:33:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 16:33:16.926300 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328d76c8-6d50-411f-bbab-40733eca2e57 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:33:16.926782 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328d76c8-6d50-411f-bbab-40733eca2e57 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:33:16.926891 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328d76c8-6d50-411f-bbab-40733eca2e57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:33:16.944905 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-328d76c8-6d50-411f-bbab-40733eca2e57 None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:33:16.946043 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-328d76c8-6d50-411f-bbab-40733eca2e57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:33:16.954758 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 683/1378] 10.222.0.57 () {58 vars in 1023 bytes} [Fri Jul 1 16:33:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 16:33:16.999181 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ae6b02-9baa-45ba-9dae-dc5026be5095 None None] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:33:17.013684 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ae6b02-9baa-45ba-9dae-dc5026be5095 None None] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:33:17.019693 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3863143c-c8c6-44c4-a6f1-ee0ea9fc20f7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a50f745b31941b7a7265c0fc852f48e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0788e6725a84e57a4174284c1845755', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:33:17.020337 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3863143c-c8c6-44c4-a6f1-ee0ea9fc20f7 service nova] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:33:17.020626 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3863143c-c8c6-44c4-a6f1-ee0ea9fc20f7 service nova] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:33:17.020908 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3863143c-c8c6-44c4-a6f1-ee0ea9fc20f7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:33:17.029308 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3863143c-c8c6-44c4-a6f1-ee0ea9fc20f7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:33:17.031023 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3863143c-c8c6-44c4-a6f1-ee0ea9fc20f7 service nova] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:33:17.034951 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=100 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:33:17.041650 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 696/1379] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 1 16:33:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 16:33:19.154730 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aedcda07-8dd1-451e-b32a-6ebca2efe6e5 None None] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:33:19.155242 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aedcda07-8dd1-451e-b32a-6ebca2efe6e5 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:33:19.155242 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aedcda07-8dd1-451e-b32a-6ebca2efe6e5 None None] PATH_INFO: `/` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:33:19.157289 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 684/1380] 10.222.0.57 () {54 vars in 926 bytes} [Fri Jul 1 16:33:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 16:33:19.162511 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7709496-e2e2-402a-aba5-04817372f523 None None] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:33:19.162962 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7709496-e2e2-402a-aba5-04817372f523 None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:33:19.163071 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7709496-e2e2-402a-aba5-04817372f523 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:33:19.180991 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.auth.core [None req-c7709496-e2e2-402a-aba5-04817372f523 None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80312) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:33:19.182174 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-c7709496-e2e2-402a-aba5-04817372f523 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:33:19.191175 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 697/1381] 10.222.0.57 () {58 vars in 1023 bytes} [Fri Jul 1 16:33:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 01 16:33:19.235436 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aedcda07-8dd1-451e-b32a-6ebca2efe6e5 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:33:19.250106 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aedcda07-8dd1-451e-b32a-6ebca2efe6e5 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:33:19.255938 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b9dee5-9a39-4e34-ad42-527c8e035699 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a50f745b31941b7a7265c0fc852f48e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0788e6725a84e57a4174284c1845755', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:33:19.256587 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b9dee5-9a39-4e34-ad42-527c8e035699 service nova] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:33:19.256876 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b9dee5-9a39-4e34-ad42-527c8e035699 service nova] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:33:19.257158 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b9dee5-9a39-4e34-ad42-527c8e035699 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:33:19.265684 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b9dee5-9a39-4e34-ad42-527c8e035699 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:33:19.267303 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b9dee5-9a39-4e34-ad42-527c8e035699 service nova] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:33:19.276774 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 685/1382] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 1 16:33:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 16:33:21.387024 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276feb8d-6341-4052-a156-9a6730121f0e None None] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:33:21.387596 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276feb8d-6341-4052-a156-9a6730121f0e None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:33:21.387596 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276feb8d-6341-4052-a156-9a6730121f0e None None] PATH_INFO: `/` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:33:21.388968 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 698/1383] 10.222.0.57 () {54 vars in 926 bytes} [Fri Jul 1 16:33:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 16:33:21.394936 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6a4780-6145-4b55-8d28-545c79608d18 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:33:21.395408 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6a4780-6145-4b55-8d28-545c79608d18 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:33:21.395512 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6a4780-6145-4b55-8d28-545c79608d18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:33:21.414042 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-bd6a4780-6145-4b55-8d28-545c79608d18 None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:33:21.415282 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-bd6a4780-6145-4b55-8d28-545c79608d18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:33:21.423546 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 686/1384] 10.222.0.57 () {58 vars in 1023 bytes} [Fri Jul 1 16:33:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 16:33:21.466080 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276feb8d-6341-4052-a156-9a6730121f0e None None] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:33:21.481708 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276feb8d-6341-4052-a156-9a6730121f0e None None] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:33:21.487097 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1464476-4197-483e-abd4-0e75faf979dc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a50f745b31941b7a7265c0fc852f48e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0788e6725a84e57a4174284c1845755', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:33:21.487740 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1464476-4197-483e-abd4-0e75faf979dc service nova] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:33:21.488019 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1464476-4197-483e-abd4-0e75faf979dc service nova] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:33:21.488294 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1464476-4197-483e-abd4-0e75faf979dc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:33:21.497283 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1464476-4197-483e-abd4-0e75faf979dc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:33:21.498817 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1464476-4197-483e-abd4-0e75faf979dc service nova] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:33:21.509175 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 699/1385] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 1 16:33:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 16:33:23.622414 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44635c8-404c-4b8b-afe1-3fc6b7c7a2be None None] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:33:23.622950 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44635c8-404c-4b8b-afe1-3fc6b7c7a2be None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:33:23.623045 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44635c8-404c-4b8b-afe1-3fc6b7c7a2be None None] PATH_INFO: `/` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:33:23.624993 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 687/1386] 10.222.0.57 () {54 vars in 926 bytes} [Fri Jul 1 16:33:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 16:33:23.630326 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2a8334-e7a9-4194-93c8-653fb4e87f4a None None] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:33:23.630863 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2a8334-e7a9-4194-93c8-653fb4e87f4a None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:33:23.630959 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2a8334-e7a9-4194-93c8-653fb4e87f4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:33:23.649633 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.auth.core [None req-0a2a8334-e7a9-4194-93c8-653fb4e87f4a None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80312) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:33:23.650904 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-0a2a8334-e7a9-4194-93c8-653fb4e87f4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:33:23.659174 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 700/1387] 10.222.0.57 () {58 vars in 1023 bytes} [Fri Jul 1 16:33:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 16:33:23.705196 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44635c8-404c-4b8b-afe1-3fc6b7c7a2be None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:33:23.720392 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44635c8-404c-4b8b-afe1-3fc6b7c7a2be None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:33:23.725998 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea49913-917d-4128-8425-f6dd575a83b1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a50f745b31941b7a7265c0fc852f48e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0788e6725a84e57a4174284c1845755', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:33:23.726674 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea49913-917d-4128-8425-f6dd575a83b1 service nova] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:33:23.726966 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea49913-917d-4128-8425-f6dd575a83b1 service nova] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:33:23.727250 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea49913-917d-4128-8425-f6dd575a83b1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:33:23.736268 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea49913-917d-4128-8425-f6dd575a83b1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:33:23.737869 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea49913-917d-4128-8425-f6dd575a83b1 service nova] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:33:23.747460 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 688/1388] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 1 16:33:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 16:33:26.144272 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83904a1-1b8b-4840-88e6-ed5ee79e5ba4 None None] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:33:26.144783 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83904a1-1b8b-4840-88e6-ed5ee79e5ba4 None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:33:26.144783 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83904a1-1b8b-4840-88e6-ed5ee79e5ba4 None None] PATH_INFO: `/` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:33:26.146143 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 701/1389] 10.222.0.57 () {54 vars in 926 bytes} [Fri Jul 1 16:33:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 01 16:33:26.151575 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa8aa06-8c6a-41f6-b75f-de517abb5bc4 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:33:26.152051 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa8aa06-8c6a-41f6-b75f-de517abb5bc4 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:33:26.152149 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa8aa06-8c6a-41f6-b75f-de517abb5bc4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:33:26.170366 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-afa8aa06-8c6a-41f6-b75f-de517abb5bc4 None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:33:26.171643 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-afa8aa06-8c6a-41f6-b75f-de517abb5bc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:33:26.179522 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 689/1390] 10.222.0.57 () {58 vars in 1023 bytes} [Fri Jul 1 16:33:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 16:33:26.229032 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83904a1-1b8b-4840-88e6-ed5ee79e5ba4 None None] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:33:26.243610 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83904a1-1b8b-4840-88e6-ed5ee79e5ba4 None None] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:33:26.249985 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a76b5d-ba19-4a81-98cc-bdd6095a662c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a50f745b31941b7a7265c0fc852f48e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0788e6725a84e57a4174284c1845755', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:33:26.250669 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a76b5d-ba19-4a81-98cc-bdd6095a662c service nova] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:33:26.250956 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a76b5d-ba19-4a81-98cc-bdd6095a662c service nova] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:33:26.251236 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a76b5d-ba19-4a81-98cc-bdd6095a662c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:33:26.259471 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a76b5d-ba19-4a81-98cc-bdd6095a662c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:33:26.261056 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a76b5d-ba19-4a81-98cc-bdd6095a662c service nova] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:33:26.272223 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 702/1391] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 1 16:33:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 16:33:28.391178 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4e913b-9cbd-416d-963e-2ce834f584a4 None None] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:33:28.391732 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4e913b-9cbd-416d-963e-2ce834f584a4 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:33:28.391732 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4e913b-9cbd-416d-963e-2ce834f584a4 None None] PATH_INFO: `/` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:33:28.393121 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 690/1392] 10.222.0.57 () {54 vars in 926 bytes} [Fri Jul 1 16:33:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 01 16:33:28.398703 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0782f6-33e8-4e30-80fc-0c3146276f56 None None] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:33:28.399095 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0782f6-33e8-4e30-80fc-0c3146276f56 None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:33:28.399279 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0782f6-33e8-4e30-80fc-0c3146276f56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:33:28.419287 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.auth.core [None req-3a0782f6-33e8-4e30-80fc-0c3146276f56 None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80312) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:33:28.420486 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-3a0782f6-33e8-4e30-80fc-0c3146276f56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:33:28.429027 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 703/1393] 10.222.0.57 () {58 vars in 1023 bytes} [Fri Jul 1 16:33:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 16:33:28.479549 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4e913b-9cbd-416d-963e-2ce834f584a4 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:33:28.494990 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4e913b-9cbd-416d-963e-2ce834f584a4 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:33:28.501611 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9019b2-a488-4b06-880e-d3585b7bbacf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a50f745b31941b7a7265c0fc852f48e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0788e6725a84e57a4174284c1845755', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:33:28.502257 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9019b2-a488-4b06-880e-d3585b7bbacf service nova] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:33:28.502575 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9019b2-a488-4b06-880e-d3585b7bbacf service nova] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:33:28.502857 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9019b2-a488-4b06-880e-d3585b7bbacf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:33:28.511513 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa9019b2-a488-4b06-880e-d3585b7bbacf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:33:28.513149 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa9019b2-a488-4b06-880e-d3585b7bbacf service nova] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:33:28.523982 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 691/1394] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 1 16:33:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 16:33:30.707761 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd73848b-44be-45e2-905f-600cb6f208b8 None None] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:33:30.708308 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd73848b-44be-45e2-905f-600cb6f208b8 None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:33:30.708433 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd73848b-44be-45e2-905f-600cb6f208b8 None None] PATH_INFO: `/` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:33:30.709749 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 704/1395] 10.222.0.57 () {54 vars in 926 bytes} [Fri Jul 1 16:33:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 16:33:30.715028 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1275dd7d-0502-45c3-abc3-0e4334bef5e5 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:33:30.715444 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1275dd7d-0502-45c3-abc3-0e4334bef5e5 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:33:30.715605 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1275dd7d-0502-45c3-abc3-0e4334bef5e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:33:30.733439 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-1275dd7d-0502-45c3-abc3-0e4334bef5e5 None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:33:30.734699 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-1275dd7d-0502-45c3-abc3-0e4334bef5e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:33:30.743179 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 692/1396] 10.222.0.57 () {58 vars in 1023 bytes} [Fri Jul 1 16:33:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 16:33:30.789465 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd73848b-44be-45e2-905f-600cb6f208b8 None None] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:33:30.804629 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd73848b-44be-45e2-905f-600cb6f208b8 None None] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:33:30.810127 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b54c217-c5fb-4c81-ac0d-21e7b527c020 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a50f745b31941b7a7265c0fc852f48e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0788e6725a84e57a4174284c1845755', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:33:30.810809 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b54c217-c5fb-4c81-ac0d-21e7b527c020 service nova] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:33:30.811631 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b54c217-c5fb-4c81-ac0d-21e7b527c020 service nova] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:33:30.811946 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b54c217-c5fb-4c81-ac0d-21e7b527c020 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:33:30.821031 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b54c217-c5fb-4c81-ac0d-21e7b527c020 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:33:30.822732 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b54c217-c5fb-4c81-ac0d-21e7b527c020 service nova] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:33:30.839847 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 705/1397] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 1 16:33:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 16:33:31.349891 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b717523d-232a-4327-b52a-6e6a1a0362c4 None None] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:33:31.350181 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b717523d-232a-4327-b52a-6e6a1a0362c4 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:33:31.350504 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b717523d-232a-4327-b52a-6e6a1a0362c4 None None] PATH_INFO: `/` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:33:31.351904 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 693/1398] 10.222.0.57 () {54 vars in 926 bytes} [Fri Jul 1 16:33:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 16:33:31.356991 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f02009-bdb4-418f-a1b4-c0d5bee954ba None None] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:33:31.357288 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f02009-bdb4-418f-a1b4-c0d5bee954ba None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:33:31.357570 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f02009-bdb4-418f-a1b4-c0d5bee954ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:33:31.375373 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.auth.core [None req-72f02009-bdb4-418f-a1b4-c0d5bee954ba None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80312) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:33:31.376575 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-72f02009-bdb4-418f-a1b4-c0d5bee954ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:33:31.384702 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 706/1399] 10.222.0.57 () {58 vars in 1023 bytes} [Fri Jul 1 16:33:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 16:33:31.443257 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a89cc2-e2fe-48fd-a563-1e6794e95fe6 None None] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:33:31.443550 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a89cc2-e2fe-48fd-a563-1e6794e95fe6 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:33:31.443834 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a89cc2-e2fe-48fd-a563-1e6794e95fe6 None None] PATH_INFO: `/` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:33:31.445216 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 694/1400] 10.222.0.57 () {54 vars in 962 bytes} [Fri Jul 1 16:33:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 16:33:31.450096 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efcf4ce-416c-4be6-9495-676525a19122 None None] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:33:31.450506 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efcf4ce-416c-4be6-9495-676525a19122 None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:33:31.450777 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efcf4ce-416c-4be6-9495-676525a19122 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:33:31.499505 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.auth.core [None req-1efcf4ce-416c-4be6-9495-676525a19122 None None] MFA Rules not processed for user `fd028715bfea43099e7c6cde429ecf92`. Rule list: `[]` (Enabled: `True`). {{(pid=80312) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:33:31.500710 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-1efcf4ce-416c-4be6-9495-676525a19122 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:33:31.569348 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 707/1401] 10.222.0.57 () {58 vars in 1059 bytes} [Fri Jul 1 16:33:31 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 16:33:31.579567 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a89cc2-e2fe-48fd-a563-1e6794e95fe6 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:33:31.595122 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a89cc2-e2fe-48fd-a563-1e6794e95fe6 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:33:31.600296 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4232c9be-b5b6-47ad-8368-e33f942015af service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd028715bfea43099e7c6cde429ecf92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0788e6725a84e57a4174284c1845755', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:33:31.601183 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4232c9be-b5b6-47ad-8368-e33f942015af service glance] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:33:31.601558 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4232c9be-b5b6-47ad-8368-e33f942015af service glance] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:33:31.601970 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4232c9be-b5b6-47ad-8368-e33f942015af service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:33:31.615078 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4232c9be-b5b6-47ad-8368-e33f942015af service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:33:31.617384 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4232c9be-b5b6-47ad-8368-e33f942015af service glance] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:33:31.630518 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 695/1402] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 1 16:33:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 16:33:32.362039 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a6c1f0-d22a-44dd-9c21-d5c49bb1fb34 None None] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:33:32.362581 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a6c1f0-d22a-44dd-9c21-d5c49bb1fb34 None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:33:32.362802 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a6c1f0-d22a-44dd-9c21-d5c49bb1fb34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:33:32.413872 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.auth.core [None req-c1a6c1f0-d22a-44dd-9c21-d5c49bb1fb34 None None] MFA Rules not processed for user `c4de1b1eeaed4fc5ae7200b2c7a90957`. Rule list: `[]` (Enabled: `True`). {{(pid=80312) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:33:32.415204 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-c1a6c1f0-d22a-44dd-9c21-d5c49bb1fb34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:33:32.456833 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=100 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:33:32.481711 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 708/1403] 10.222.0.57 () {58 vars in 1009 bytes} [Fri Jul 1 16:33:32 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 16:33:32.523768 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a27b0f-bf19-4de1-bd1a-e91f87d845cb None None] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:33:32.524263 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a27b0f-bf19-4de1-bd1a-e91f87d845cb None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:33:32.524360 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a27b0f-bf19-4de1-bd1a-e91f87d845cb None None] PATH_INFO: `/` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:33:32.525816 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 696/1404] 10.222.0.57 () {54 vars in 942 bytes} [Fri Jul 1 16:33:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 16:33:32.532820 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004674d3-851f-40c3-9feb-e959375ebfa1 None None] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:33:32.533114 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004674d3-851f-40c3-9feb-e959375ebfa1 None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:33:32.533394 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004674d3-851f-40c3-9feb-e959375ebfa1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:33:32.585644 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.auth.core [None req-004674d3-851f-40c3-9feb-e959375ebfa1 None None] MFA Rules not processed for user `859fe410c2d2450fb803c6244839f27c`. Rule list: `[]` (Enabled: `True`). {{(pid=80312) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:33:32.586895 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-004674d3-851f-40c3-9feb-e959375ebfa1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:33:32.663034 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 709/1405] 10.222.0.57 () {58 vars in 1039 bytes} [Fri Jul 1 16:33:32 2022] POST /identity/v3/auth/tokens => generated 3294 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 16:33:32.673349 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a27b0f-bf19-4de1-bd1a-e91f87d845cb None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:33:32.681067 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=61 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:33:32.688250 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a27b0f-bf19-4de1-bd1a-e91f87d845cb None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:33:32.693552 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb401f5-546a-410b-9744-b6c1a6d5c317 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '859fe410c2d2450fb803c6244839f27c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0788e6725a84e57a4174284c1845755', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:33:32.694259 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb401f5-546a-410b-9744-b6c1a6d5c317 service swift] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:33:32.694649 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb401f5-546a-410b-9744-b6c1a6d5c317 service swift] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:33:32.694993 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb401f5-546a-410b-9744-b6c1a6d5c317 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:33:32.704002 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb401f5-546a-410b-9744-b6c1a6d5c317 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:33:32.705630 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb401f5-546a-410b-9744-b6c1a6d5c317 service swift] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:33:32.713996 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 697/1406] 10.222.0.57 () {60 vars in 1384 bytes} [Fri Jul 1 16:33:32 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 01 16:33:32.961458 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7646cab8-2790-4b17-82bd-bf9aef3530d0 None None] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:33:32.961785 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7646cab8-2790-4b17-82bd-bf9aef3530d0 None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:33:32.962023 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7646cab8-2790-4b17-82bd-bf9aef3530d0 None None] PATH_INFO: `/` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:33:32.968859 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bb8adf-50c3-440f-a431-c16d9e881149 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:33:32.969152 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bb8adf-50c3-440f-a431-c16d9e881149 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:33:32.969432 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bb8adf-50c3-440f-a431-c16d9e881149 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:33:32.974369 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 710/1407] 10.222.0.57 () {54 vars in 926 bytes} [Fri Jul 1 16:33:32 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 16:33:32.988848 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-a5bb8adf-50c3-440f-a431-c16d9e881149 None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:33:32.990331 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-a5bb8adf-50c3-440f-a431-c16d9e881149 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:33:33.018514 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 698/1408] 10.222.0.57 () {58 vars in 1023 bytes} [Fri Jul 1 16:33:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 16:33:33.048914 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7646cab8-2790-4b17-82bd-bf9aef3530d0 None None] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:33:33.066585 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7646cab8-2790-4b17-82bd-bf9aef3530d0 None None] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:33:33.077867 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292b4e2a-d616-4c20-8250-dd8ae262c57d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a50f745b31941b7a7265c0fc852f48e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0788e6725a84e57a4174284c1845755', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:33:33.078729 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292b4e2a-d616-4c20-8250-dd8ae262c57d service nova] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:33:33.079136 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292b4e2a-d616-4c20-8250-dd8ae262c57d service nova] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:33:33.079562 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292b4e2a-d616-4c20-8250-dd8ae262c57d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:33:33.089444 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-292b4e2a-d616-4c20-8250-dd8ae262c57d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:33:33.091376 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-292b4e2a-d616-4c20-8250-dd8ae262c57d service nova] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:33:33.103795 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 711/1409] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 1 16:33:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 16:33:42.987149 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=8 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:33:43.101378 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=59 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:33:55.305375 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40f76ce-a394-45d1-9379-c2b310d661da None None] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:33:55.305942 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40f76ce-a394-45d1-9379-c2b310d661da None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:33:55.306048 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40f76ce-a394-45d1-9379-c2b310d661da None None] PATH_INFO: `/` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:33:55.307436 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 699/1410] 10.222.0.57 () {54 vars in 918 bytes} [Fri Jul 1 16:33:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 16:33:55.313119 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be918c3e-9d72-45ca-a9e7-3c6e780991d1 None None] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:33:55.313578 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be918c3e-9d72-45ca-a9e7-3c6e780991d1 None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:33:55.313688 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be918c3e-9d72-45ca-a9e7-3c6e780991d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:33:55.332108 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.auth.core [None req-be918c3e-9d72-45ca-a9e7-3c6e780991d1 None None] MFA Rules not processed for user `bbc38af18dec4e9b8143df23af6aad43`. Rule list: `[]` (Enabled: `True`). {{(pid=80312) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:33:55.333294 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-be918c3e-9d72-45ca-a9e7-3c6e780991d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:33:55.342266 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 712/1411] 10.222.0.57 () {58 vars in 1015 bytes} [Fri Jul 1 16:33:55 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 16:33:55.356068 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79a9884-9928-48a6-a89e-5ef7dc73d5a6 None None] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:33:55.356460 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79a9884-9928-48a6-a89e-5ef7dc73d5a6 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:33:55.356968 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79a9884-9928-48a6-a89e-5ef7dc73d5a6 None None] PATH_INFO: `/` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:33:55.358379 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 700/1412] 10.222.0.57 () {54 vars in 918 bytes} [Fri Jul 1 16:33:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 16:33:55.362944 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc3e27d-42da-4970-adb5-181438b9ae32 None None] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:33:55.363232 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc3e27d-42da-4970-adb5-181438b9ae32 None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:33:55.363509 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc3e27d-42da-4970-adb5-181438b9ae32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:33:55.380880 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.auth.core [None req-fbc3e27d-42da-4970-adb5-181438b9ae32 None None] MFA Rules not processed for user `7a50f745b31941b7a7265c0fc852f48e`. Rule list: `[]` (Enabled: `True`). {{(pid=80312) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:33:55.382167 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-fbc3e27d-42da-4970-adb5-181438b9ae32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:33:55.391550 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 713/1413] 10.222.0.57 () {58 vars in 1015 bytes} [Fri Jul 1 16:33:55 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 01 16:33:57.915148 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0b4a4b-a143-4b7b-ba05-f1ea0f7e33c6 None None] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:33:57.915676 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0b4a4b-a143-4b7b-ba05-f1ea0f7e33c6 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:33:57.915676 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0b4a4b-a143-4b7b-ba05-f1ea0f7e33c6 None None] PATH_INFO: `/` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:33:57.917219 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 701/1414] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jul 1 16:33:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 16:33:57.922612 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c69b81-601a-424e-861a-722116c92eaa None None] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:33:57.923059 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c69b81-601a-424e-861a-722116c92eaa None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:33:57.923164 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c69b81-601a-424e-861a-722116c92eaa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:33:57.941557 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.auth.core [None req-98c69b81-601a-424e-861a-722116c92eaa None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80312) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:33:57.943187 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-98c69b81-601a-424e-861a-722116c92eaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:33:57.951900 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 714/1415] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jul 1 16:33:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 16:33:57.995786 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0b4a4b-a143-4b7b-ba05-f1ea0f7e33c6 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:33:58.011090 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0b4a4b-a143-4b7b-ba05-f1ea0f7e33c6 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:33:58.017603 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25fbcb89-8cfe-4e48-af33-d8fb4d4ccfab service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a50f745b31941b7a7265c0fc852f48e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0788e6725a84e57a4174284c1845755', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:33:58.018236 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fbcb89-8cfe-4e48-af33-d8fb4d4ccfab service nova] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:33:58.018560 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fbcb89-8cfe-4e48-af33-d8fb4d4ccfab service nova] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:33:58.018839 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fbcb89-8cfe-4e48-af33-d8fb4d4ccfab service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:33:58.026895 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25fbcb89-8cfe-4e48-af33-d8fb4d4ccfab service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:33:58.029028 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25fbcb89-8cfe-4e48-af33-d8fb4d4ccfab service nova] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:33:58.038295 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 702/1416] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 1 16:33:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 16:33:58.953854 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e8294c-4e13-4724-b895-9258383f3e3b None None] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:33:58.954423 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e8294c-4e13-4724-b895-9258383f3e3b None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:33:58.954423 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e8294c-4e13-4724-b895-9258383f3e3b None None] PATH_INFO: `/` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:33:58.955811 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 715/1417] 127.0.0.1 () {54 vars in 901 bytes} [Fri Jul 1 16:33:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 16:33:58.960834 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bb2bb5-dca1-4097-aa7e-070054d40e39 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:33:58.961304 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bb2bb5-dca1-4097-aa7e-070054d40e39 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:33:58.961439 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bb2bb5-dca1-4097-aa7e-070054d40e39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:33:58.980756 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-44bb2bb5-dca1-4097-aa7e-070054d40e39 None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:33:58.981984 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-44bb2bb5-dca1-4097-aa7e-070054d40e39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:33:58.989675 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 703/1418] 127.0.0.1 () {58 vars in 998 bytes} [Fri Jul 1 16:33:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 01 16:33:59.151448 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e8294c-4e13-4724-b895-9258383f3e3b None None] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:33:59.166213 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e8294c-4e13-4724-b895-9258383f3e3b None None] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:33:59.171794 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eae92fd-2477-454a-b9d9-e257c166b27a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a50f745b31941b7a7265c0fc852f48e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0788e6725a84e57a4174284c1845755', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:33:59.172421 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eae92fd-2477-454a-b9d9-e257c166b27a service nova] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:33:59.172701 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eae92fd-2477-454a-b9d9-e257c166b27a service nova] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:33:59.172972 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eae92fd-2477-454a-b9d9-e257c166b27a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:33:59.180846 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eae92fd-2477-454a-b9d9-e257c166b27a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:33:59.182439 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eae92fd-2477-454a-b9d9-e257c166b27a service nova] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:33:59.192328 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 716/1419] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 1 16:33:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 16:34:00.083956 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332046ef-83a6-41bf-a249-5fca6b2ca8de None None] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:34:00.084524 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332046ef-83a6-41bf-a249-5fca6b2ca8de None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:34:00.084524 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332046ef-83a6-41bf-a249-5fca6b2ca8de None None] PATH_INFO: `/` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:34:00.085887 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 704/1420] 127.0.0.1 () {54 vars in 901 bytes} [Fri Jul 1 16:34:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 16:34:00.091565 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf03a61d-f291-4254-8a50-16daf9ae3e72 None None] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:34:00.091982 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf03a61d-f291-4254-8a50-16daf9ae3e72 None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:34:00.092134 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf03a61d-f291-4254-8a50-16daf9ae3e72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:34:00.110826 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.auth.core [None req-cf03a61d-f291-4254-8a50-16daf9ae3e72 None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80312) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:34:00.111962 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-cf03a61d-f291-4254-8a50-16daf9ae3e72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:34:00.119773 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 717/1421] 127.0.0.1 () {58 vars in 998 bytes} [Fri Jul 1 16:34:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 16:34:00.272272 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332046ef-83a6-41bf-a249-5fca6b2ca8de None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:34:00.287354 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332046ef-83a6-41bf-a249-5fca6b2ca8de None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:34:00.292679 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1321b4-c3ab-4915-8b8d-ea5319c68027 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a50f745b31941b7a7265c0fc852f48e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0788e6725a84e57a4174284c1845755', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:34:00.293333 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1321b4-c3ab-4915-8b8d-ea5319c68027 service nova] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:34:00.294400 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1321b4-c3ab-4915-8b8d-ea5319c68027 service nova] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:34:00.295117 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1321b4-c3ab-4915-8b8d-ea5319c68027 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:34:00.303113 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a1321b4-c3ab-4915-8b8d-ea5319c68027 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:34:00.304712 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a1321b4-c3ab-4915-8b8d-ea5319c68027 service nova] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:34:00.314657 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 705/1422] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 1 16:34:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 16:34:01.195402 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb6578c-8ed1-4049-ba3b-ddad9c590f62 None None] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:34:01.195947 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb6578c-8ed1-4049-ba3b-ddad9c590f62 None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:34:01.195947 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb6578c-8ed1-4049-ba3b-ddad9c590f62 None None] PATH_INFO: `/` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:34:01.197247 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 718/1423] 127.0.0.1 () {54 vars in 901 bytes} [Fri Jul 1 16:34:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 16:34:01.202860 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c1ceab-6206-4fb7-8fa0-3f87e8504362 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:34:01.203254 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c1ceab-6206-4fb7-8fa0-3f87e8504362 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:34:01.203422 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c1ceab-6206-4fb7-8fa0-3f87e8504362 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:34:01.223945 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-94c1ceab-6206-4fb7-8fa0-3f87e8504362 None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:34:01.225148 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-94c1ceab-6206-4fb7-8fa0-3f87e8504362 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:34:01.233470 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 706/1424] 127.0.0.1 () {58 vars in 998 bytes} [Fri Jul 1 16:34:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 16:34:01.385851 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb6578c-8ed1-4049-ba3b-ddad9c590f62 None None] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:34:01.401286 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb6578c-8ed1-4049-ba3b-ddad9c590f62 None None] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:34:01.407214 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec52c9b-2114-4380-887f-8a5452aa6088 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a50f745b31941b7a7265c0fc852f48e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0788e6725a84e57a4174284c1845755', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:34:01.407850 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec52c9b-2114-4380-887f-8a5452aa6088 service nova] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:34:01.408133 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec52c9b-2114-4380-887f-8a5452aa6088 service nova] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:34:01.408411 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec52c9b-2114-4380-887f-8a5452aa6088 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:34:01.416385 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec52c9b-2114-4380-887f-8a5452aa6088 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:34:01.417955 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec52c9b-2114-4380-887f-8a5452aa6088 service nova] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:34:01.427501 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 719/1425] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 1 16:34:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 16:34:02.298914 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c3c055-f9e8-45dd-a8f6-a9be046f9cf0 None None] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:34:02.299329 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c3c055-f9e8-45dd-a8f6-a9be046f9cf0 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:34:02.299644 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c3c055-f9e8-45dd-a8f6-a9be046f9cf0 None None] PATH_INFO: `/` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:34:02.300866 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 707/1426] 127.0.0.1 () {54 vars in 901 bytes} [Fri Jul 1 16:34:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 16:34:02.305977 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2806bb54-5307-4f5c-b225-c3bd6361fc5d None None] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:34:02.306413 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2806bb54-5307-4f5c-b225-c3bd6361fc5d None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:34:02.306568 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2806bb54-5307-4f5c-b225-c3bd6361fc5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:34:02.326103 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.auth.core [None req-2806bb54-5307-4f5c-b225-c3bd6361fc5d None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80312) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:34:02.327353 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-2806bb54-5307-4f5c-b225-c3bd6361fc5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:34:02.335104 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 720/1427] 127.0.0.1 () {58 vars in 998 bytes} [Fri Jul 1 16:34:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 16:34:02.487793 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c3c055-f9e8-45dd-a8f6-a9be046f9cf0 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:34:02.502718 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c3c055-f9e8-45dd-a8f6-a9be046f9cf0 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:34:02.508183 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b420fe77-7e85-4569-a061-95b6b9b68683 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a50f745b31941b7a7265c0fc852f48e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0788e6725a84e57a4174284c1845755', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:34:02.508817 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b420fe77-7e85-4569-a061-95b6b9b68683 service nova] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:34:02.509102 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b420fe77-7e85-4569-a061-95b6b9b68683 service nova] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:34:02.509381 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b420fe77-7e85-4569-a061-95b6b9b68683 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:34:02.518148 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b420fe77-7e85-4569-a061-95b6b9b68683 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:34:02.519752 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b420fe77-7e85-4569-a061-95b6b9b68683 service nova] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:34:02.529005 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 708/1428] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 1 16:34:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 16:34:03.348531 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45347d7a-70ee-41ab-aa0e-6aa936568c14 None None] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:34:03.349100 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45347d7a-70ee-41ab-aa0e-6aa936568c14 None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:34:03.349100 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45347d7a-70ee-41ab-aa0e-6aa936568c14 None None] PATH_INFO: `/` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:34:03.350518 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 721/1429] 127.0.0.1 () {54 vars in 901 bytes} [Fri Jul 1 16:34:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 16:34:03.355615 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567ecf04-a9c4-45c9-85f3-7d2c9c4b81d0 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:34:03.356042 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567ecf04-a9c4-45c9-85f3-7d2c9c4b81d0 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:34:03.356340 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567ecf04-a9c4-45c9-85f3-7d2c9c4b81d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:34:03.375401 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-567ecf04-a9c4-45c9-85f3-7d2c9c4b81d0 None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:34:03.376600 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-567ecf04-a9c4-45c9-85f3-7d2c9c4b81d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:34:03.384306 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 709/1430] 127.0.0.1 () {58 vars in 998 bytes} [Fri Jul 1 16:34:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 16:34:03.536159 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45347d7a-70ee-41ab-aa0e-6aa936568c14 None None] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:34:03.555469 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45347d7a-70ee-41ab-aa0e-6aa936568c14 None None] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:34:03.562331 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7aa7835-752a-40b4-b053-c580ed7f1fb7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a50f745b31941b7a7265c0fc852f48e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0788e6725a84e57a4174284c1845755', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:34:03.563003 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7aa7835-752a-40b4-b053-c580ed7f1fb7 service nova] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:34:03.563287 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7aa7835-752a-40b4-b053-c580ed7f1fb7 service nova] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:34:03.563564 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7aa7835-752a-40b4-b053-c580ed7f1fb7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:34:03.572130 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7aa7835-752a-40b4-b053-c580ed7f1fb7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:34:03.573711 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7aa7835-752a-40b4-b053-c580ed7f1fb7 service nova] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:34:03.584049 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 722/1431] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 1 16:34:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 16:34:04.408820 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf828e9-028c-4e2e-97fd-e6333284f93b None None] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:34:04.409373 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf828e9-028c-4e2e-97fd-e6333284f93b None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:34:04.409373 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf828e9-028c-4e2e-97fd-e6333284f93b None None] PATH_INFO: `/` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:34:04.411142 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 710/1432] 127.0.0.1 () {54 vars in 901 bytes} [Fri Jul 1 16:34:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 16:34:04.415721 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9e5ce0-cde7-4e02-834e-bd44dcb9a80b None None] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:34:04.416199 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9e5ce0-cde7-4e02-834e-bd44dcb9a80b None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:34:04.416309 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9e5ce0-cde7-4e02-834e-bd44dcb9a80b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:34:04.435053 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.auth.core [None req-ee9e5ce0-cde7-4e02-834e-bd44dcb9a80b None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80312) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:34:04.436226 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-ee9e5ce0-cde7-4e02-834e-bd44dcb9a80b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:34:04.444224 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 723/1433] 127.0.0.1 () {58 vars in 998 bytes} [Fri Jul 1 16:34:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 16:34:04.595757 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf828e9-028c-4e2e-97fd-e6333284f93b None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:34:04.610723 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf828e9-028c-4e2e-97fd-e6333284f93b None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:34:04.616018 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61053c08-1e40-4668-b315-68a05f93df35 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a50f745b31941b7a7265c0fc852f48e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0788e6725a84e57a4174284c1845755', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:34:04.616644 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61053c08-1e40-4668-b315-68a05f93df35 service nova] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:34:04.616925 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61053c08-1e40-4668-b315-68a05f93df35 service nova] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:34:04.617198 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61053c08-1e40-4668-b315-68a05f93df35 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:34:04.625441 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61053c08-1e40-4668-b315-68a05f93df35 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:34:04.627043 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61053c08-1e40-4668-b315-68a05f93df35 service nova] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:34:04.636269 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 711/1434] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 1 16:34:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 16:34:05.445924 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5319ddd-9f9d-4ae8-8f41-c62fcb141314 None None] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:34:05.446491 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5319ddd-9f9d-4ae8-8f41-c62fcb141314 None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:34:05.446491 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5319ddd-9f9d-4ae8-8f41-c62fcb141314 None None] PATH_INFO: `/` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:34:05.447875 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 724/1435] 127.0.0.1 () {54 vars in 901 bytes} [Fri Jul 1 16:34:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 16:34:05.453419 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3de90f1-b01d-4efc-bf15-26a997cb999e None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:34:05.453862 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3de90f1-b01d-4efc-bf15-26a997cb999e None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:34:05.453983 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3de90f1-b01d-4efc-bf15-26a997cb999e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:34:05.471609 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-a3de90f1-b01d-4efc-bf15-26a997cb999e None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:34:05.472846 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-a3de90f1-b01d-4efc-bf15-26a997cb999e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:34:05.480600 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 712/1436] 127.0.0.1 () {58 vars in 998 bytes} [Fri Jul 1 16:34:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 16:34:05.632539 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5319ddd-9f9d-4ae8-8f41-c62fcb141314 None None] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:34:05.647261 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5319ddd-9f9d-4ae8-8f41-c62fcb141314 None None] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:34:05.653266 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb059b9-b675-4611-9c94-e7dda0a4054e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a50f745b31941b7a7265c0fc852f48e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0788e6725a84e57a4174284c1845755', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:34:05.654003 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb059b9-b675-4611-9c94-e7dda0a4054e service nova] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:34:05.654282 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb059b9-b675-4611-9c94-e7dda0a4054e service nova] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:34:05.654592 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb059b9-b675-4611-9c94-e7dda0a4054e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:34:05.663174 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb059b9-b675-4611-9c94-e7dda0a4054e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:34:05.664764 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb059b9-b675-4611-9c94-e7dda0a4054e service nova] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:34:05.674304 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 725/1437] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 1 16:34:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 16:34:06.486462 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e2dfca-f56e-4b2a-ad28-c5a5f24d48b1 None None] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:34:06.487068 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e2dfca-f56e-4b2a-ad28-c5a5f24d48b1 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:34:06.487068 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e2dfca-f56e-4b2a-ad28-c5a5f24d48b1 None None] PATH_INFO: `/` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:34:06.488383 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 713/1438] 127.0.0.1 () {54 vars in 901 bytes} [Fri Jul 1 16:34:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 16:34:06.493405 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb35af7-dcd9-4a07-82f5-04063d025365 None None] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:34:06.493821 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb35af7-dcd9-4a07-82f5-04063d025365 None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:34:06.494004 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb35af7-dcd9-4a07-82f5-04063d025365 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:34:06.512840 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.auth.core [None req-4fb35af7-dcd9-4a07-82f5-04063d025365 None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80312) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:34:06.514039 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-4fb35af7-dcd9-4a07-82f5-04063d025365 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:34:06.521952 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 726/1439] 127.0.0.1 () {58 vars in 998 bytes} [Fri Jul 1 16:34:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 16:34:06.674698 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e2dfca-f56e-4b2a-ad28-c5a5f24d48b1 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:34:06.689724 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e2dfca-f56e-4b2a-ad28-c5a5f24d48b1 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:34:06.695257 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb539f38-0fe1-436a-8158-c5d9ae40dbeb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a50f745b31941b7a7265c0fc852f48e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0788e6725a84e57a4174284c1845755', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:34:06.695892 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb539f38-0fe1-436a-8158-c5d9ae40dbeb service nova] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:34:06.696195 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb539f38-0fe1-436a-8158-c5d9ae40dbeb service nova] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:34:06.696456 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb539f38-0fe1-436a-8158-c5d9ae40dbeb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:34:06.704891 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb539f38-0fe1-436a-8158-c5d9ae40dbeb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:34:06.706488 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb539f38-0fe1-436a-8158-c5d9ae40dbeb service nova] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:34:06.716447 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 714/1440] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 1 16:34:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 16:34:08.678102 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23b81d8-af5c-4d70-bf56-d44f78a56ecf None None] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:34:08.678677 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23b81d8-af5c-4d70-bf56-d44f78a56ecf None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:34:08.678677 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23b81d8-af5c-4d70-bf56-d44f78a56ecf None None] PATH_INFO: `/` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:34:08.679992 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 727/1441] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jul 1 16:34:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 16:34:08.686102 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e4102d-ce99-42ca-b951-8c4af2c131c3 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:34:08.686538 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e4102d-ce99-42ca-b951-8c4af2c131c3 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:34:08.686679 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e4102d-ce99-42ca-b951-8c4af2c131c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:34:08.705072 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-12e4102d-ce99-42ca-b951-8c4af2c131c3 None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:34:08.706299 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-12e4102d-ce99-42ca-b951-8c4af2c131c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:34:08.714824 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 715/1442] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jul 1 16:34:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 16:34:08.754986 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23b81d8-af5c-4d70-bf56-d44f78a56ecf None None] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:34:08.766133 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23b81d8-af5c-4d70-bf56-d44f78a56ecf None None] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:34:08.769597 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3e939a-060b-43a4-ab1a-73f3350cd5af service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ed200ce29684c5f80354fa9a8638fe2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0788e6725a84e57a4174284c1845755', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:34:08.770230 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3e939a-060b-43a4-ab1a-73f3350cd5af service neutron] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:34:08.770555 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3e939a-060b-43a4-ab1a-73f3350cd5af service neutron] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:34:08.770836 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3e939a-060b-43a4-ab1a-73f3350cd5af service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:34:08.781629 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a3e939a-060b-43a4-ab1a-73f3350cd5af service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:34:08.784220 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a3e939a-060b-43a4-ab1a-73f3350cd5af service neutron] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:34:08.796414 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 728/1443] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 1 16:34:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 16:34:10.974147 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9d79a6-9a2d-48f1-958b-7dd967c32bef None None] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:34:10.974683 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9d79a6-9a2d-48f1-958b-7dd967c32bef None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:34:10.974683 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9d79a6-9a2d-48f1-958b-7dd967c32bef None None] PATH_INFO: `/` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:34:10.976190 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 716/1444] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jul 1 16:34:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 16:34:10.981564 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3575835-f6f7-4a65-b366-8843f8f3d604 None None] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:34:10.981981 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3575835-f6f7-4a65-b366-8843f8f3d604 None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:34:10.982160 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3575835-f6f7-4a65-b366-8843f8f3d604 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:34:11.000146 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.auth.core [None req-a3575835-f6f7-4a65-b366-8843f8f3d604 None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80312) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:34:11.001331 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-a3575835-f6f7-4a65-b366-8843f8f3d604 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:34:11.009261 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 729/1445] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jul 1 16:34:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 16:34:11.048727 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9d79a6-9a2d-48f1-958b-7dd967c32bef None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:34:11.059968 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9d79a6-9a2d-48f1-958b-7dd967c32bef None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:34:11.063480 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b901a60-6cbc-480f-8889-48104427816e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ed200ce29684c5f80354fa9a8638fe2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0788e6725a84e57a4174284c1845755', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:34:11.064109 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b901a60-6cbc-480f-8889-48104427816e service neutron] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:34:11.064394 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b901a60-6cbc-480f-8889-48104427816e service neutron] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:34:11.064673 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b901a60-6cbc-480f-8889-48104427816e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:34:11.073809 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b901a60-6cbc-480f-8889-48104427816e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:34:11.076317 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b901a60-6cbc-480f-8889-48104427816e service neutron] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:34:11.085749 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 717/1446] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 1 16:34:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 16:34:13.565048 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ed8d19-a7a7-4ac4-bff1-49f865dda299 None None] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:34:13.565612 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ed8d19-a7a7-4ac4-bff1-49f865dda299 None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:34:13.565612 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ed8d19-a7a7-4ac4-bff1-49f865dda299 None None] PATH_INFO: `/` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:34:13.567319 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 730/1447] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jul 1 16:34:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 16:34:13.573160 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec821d6-a5b9-46ca-ab4b-93915fb3ffec None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:34:13.573609 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec821d6-a5b9-46ca-ab4b-93915fb3ffec None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:34:13.573734 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec821d6-a5b9-46ca-ab4b-93915fb3ffec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:34:13.592219 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-aec821d6-a5b9-46ca-ab4b-93915fb3ffec None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:34:13.593419 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-aec821d6-a5b9-46ca-ab4b-93915fb3ffec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:34:13.601221 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 718/1448] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jul 1 16:34:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 16:34:13.639997 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ed8d19-a7a7-4ac4-bff1-49f865dda299 None None] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:34:13.650698 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ed8d19-a7a7-4ac4-bff1-49f865dda299 None None] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:34:13.654891 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba057a2-5b58-4b54-affa-18dae9d05a37 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ed200ce29684c5f80354fa9a8638fe2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0788e6725a84e57a4174284c1845755', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:34:13.655550 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba057a2-5b58-4b54-affa-18dae9d05a37 service neutron] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:34:13.655832 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba057a2-5b58-4b54-affa-18dae9d05a37 service neutron] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:34:13.656108 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba057a2-5b58-4b54-affa-18dae9d05a37 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:34:13.664930 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba057a2-5b58-4b54-affa-18dae9d05a37 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:34:13.666366 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba057a2-5b58-4b54-affa-18dae9d05a37 service neutron] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:34:13.676605 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 731/1449] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 1 16:34:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 16:34:15.942880 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07544d8-a08e-4d5f-88df-b32c9ad3bb4a None None] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:34:15.943454 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07544d8-a08e-4d5f-88df-b32c9ad3bb4a None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:34:15.943454 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07544d8-a08e-4d5f-88df-b32c9ad3bb4a None None] PATH_INFO: `/` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:34:15.944792 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 719/1450] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jul 1 16:34:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 16:34:15.950358 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649ec38f-22f5-4416-839f-205ed69fe548 None None] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:34:15.950785 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649ec38f-22f5-4416-839f-205ed69fe548 None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:34:15.950921 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649ec38f-22f5-4416-839f-205ed69fe548 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:34:15.969074 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.auth.core [None req-649ec38f-22f5-4416-839f-205ed69fe548 None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80312) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:34:15.970268 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-649ec38f-22f5-4416-839f-205ed69fe548 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:34:15.978051 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 732/1451] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jul 1 16:34:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 16:34:16.017086 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07544d8-a08e-4d5f-88df-b32c9ad3bb4a None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:34:16.028362 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07544d8-a08e-4d5f-88df-b32c9ad3bb4a None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:34:16.032059 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21641835-bf2e-4c06-a850-e1c71801cae0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ed200ce29684c5f80354fa9a8638fe2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0788e6725a84e57a4174284c1845755', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:34:16.032693 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21641835-bf2e-4c06-a850-e1c71801cae0 service neutron] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:34:16.032979 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21641835-bf2e-4c06-a850-e1c71801cae0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:34:16.033260 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21641835-bf2e-4c06-a850-e1c71801cae0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:34:16.042203 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21641835-bf2e-4c06-a850-e1c71801cae0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:34:16.044641 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21641835-bf2e-4c06-a850-e1c71801cae0 service neutron] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:34:16.054015 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 720/1452] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 1 16:34:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 16:34:18.160398 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ee28f6-fbc1-489a-a45d-c209efecfe67 None None] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:34:18.160932 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ee28f6-fbc1-489a-a45d-c209efecfe67 None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:34:18.160932 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ee28f6-fbc1-489a-a45d-c209efecfe67 None None] PATH_INFO: `/` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:34:18.162237 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 733/1453] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jul 1 16:34:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 16:34:18.169370 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67bb9945-ec3d-4035-958b-c179db70aacb None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:34:18.169869 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67bb9945-ec3d-4035-958b-c179db70aacb None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:34:18.170365 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67bb9945-ec3d-4035-958b-c179db70aacb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:34:18.191179 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-67bb9945-ec3d-4035-958b-c179db70aacb None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:34:18.192643 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-67bb9945-ec3d-4035-958b-c179db70aacb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:34:18.201866 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 721/1454] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jul 1 16:34:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 16:34:18.244115 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ee28f6-fbc1-489a-a45d-c209efecfe67 None None] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:34:18.255327 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ee28f6-fbc1-489a-a45d-c209efecfe67 None None] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:34:18.258824 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4876439c-98b8-4b11-9041-0847d39c5276 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ed200ce29684c5f80354fa9a8638fe2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0788e6725a84e57a4174284c1845755', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:34:18.259459 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4876439c-98b8-4b11-9041-0847d39c5276 service neutron] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:34:18.259742 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4876439c-98b8-4b11-9041-0847d39c5276 service neutron] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:34:18.260019 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4876439c-98b8-4b11-9041-0847d39c5276 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:34:18.269241 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4876439c-98b8-4b11-9041-0847d39c5276 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:34:18.270711 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4876439c-98b8-4b11-9041-0847d39c5276 service neutron] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:34:18.280834 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 734/1455] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 1 16:34:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 16:34:20.427611 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95124210-c55f-4dae-8657-daa9df552f1d None None] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:34:20.428165 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95124210-c55f-4dae-8657-daa9df552f1d None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:34:20.428165 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95124210-c55f-4dae-8657-daa9df552f1d None None] PATH_INFO: `/` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:34:20.429586 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 722/1456] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jul 1 16:34:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 16:34:20.435511 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceeddf67-cb97-45fb-a66a-5cbc71014eae None None] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:34:20.435915 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceeddf67-cb97-45fb-a66a-5cbc71014eae None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:34:20.436076 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceeddf67-cb97-45fb-a66a-5cbc71014eae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:34:20.454336 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.auth.core [None req-ceeddf67-cb97-45fb-a66a-5cbc71014eae None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80312) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:34:20.455556 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-ceeddf67-cb97-45fb-a66a-5cbc71014eae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:34:20.463565 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 735/1457] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jul 1 16:34:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 16:34:20.502643 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95124210-c55f-4dae-8657-daa9df552f1d None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:34:20.513337 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95124210-c55f-4dae-8657-daa9df552f1d None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:34:20.516859 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c31badc-5362-4873-a43d-a38225583746 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ed200ce29684c5f80354fa9a8638fe2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0788e6725a84e57a4174284c1845755', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:34:20.517495 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c31badc-5362-4873-a43d-a38225583746 service neutron] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:34:20.517782 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c31badc-5362-4873-a43d-a38225583746 service neutron] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:34:20.518067 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c31badc-5362-4873-a43d-a38225583746 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:34:20.526910 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c31badc-5362-4873-a43d-a38225583746 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:34:20.528992 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c31badc-5362-4873-a43d-a38225583746 service neutron] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:34:20.538510 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 723/1458] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 1 16:34:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 16:34:22.643688 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942491a6-bf06-4aa3-865f-e56b2d57a2a1 None None] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:34:22.644323 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942491a6-bf06-4aa3-865f-e56b2d57a2a1 None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:34:22.644323 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942491a6-bf06-4aa3-865f-e56b2d57a2a1 None None] PATH_INFO: `/` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:34:22.645649 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 736/1459] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jul 1 16:34:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 16:34:22.651052 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5096f91c-cf5f-412e-a950-3e334552b791 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:34:22.651497 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5096f91c-cf5f-412e-a950-3e334552b791 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:34:22.651606 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5096f91c-cf5f-412e-a950-3e334552b791 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:34:22.669454 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-5096f91c-cf5f-412e-a950-3e334552b791 None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:34:22.670702 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-5096f91c-cf5f-412e-a950-3e334552b791 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:34:22.678517 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 724/1460] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jul 1 16:34:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 16:34:22.718203 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942491a6-bf06-4aa3-865f-e56b2d57a2a1 None None] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:34:22.729001 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942491a6-bf06-4aa3-865f-e56b2d57a2a1 None None] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:34:22.732895 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be6fec5-4531-4d0a-af96-d0b0b0c12b1d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ed200ce29684c5f80354fa9a8638fe2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0788e6725a84e57a4174284c1845755', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:34:22.733532 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be6fec5-4531-4d0a-af96-d0b0b0c12b1d service neutron] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:34:22.733820 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be6fec5-4531-4d0a-af96-d0b0b0c12b1d service neutron] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:34:22.734100 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be6fec5-4531-4d0a-af96-d0b0b0c12b1d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:34:22.742963 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be6fec5-4531-4d0a-af96-d0b0b0c12b1d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:34:22.744461 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be6fec5-4531-4d0a-af96-d0b0b0c12b1d service neutron] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:34:22.754308 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 737/1461] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 1 16:34:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 16:34:24.865976 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcc894b-db17-4acf-9972-13c846eeac62 None None] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:34:24.866465 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcc894b-db17-4acf-9972-13c846eeac62 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:34:24.866562 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcc894b-db17-4acf-9972-13c846eeac62 None None] PATH_INFO: `/` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:34:24.867996 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 725/1462] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jul 1 16:34:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 16:34:24.873322 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225f34db-2d67-4d29-bdc0-5a2828b6ad39 None None] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:34:24.873763 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225f34db-2d67-4d29-bdc0-5a2828b6ad39 None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:34:24.873868 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225f34db-2d67-4d29-bdc0-5a2828b6ad39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:34:24.879970 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=79 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:34:24.893175 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.auth.core [None req-225f34db-2d67-4d29-bdc0-5a2828b6ad39 None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80312) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:34:24.894396 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-225f34db-2d67-4d29-bdc0-5a2828b6ad39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:34:24.902218 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 738/1463] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jul 1 16:34:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 16:34:24.941207 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bcc894b-db17-4acf-9972-13c846eeac62 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:34:24.946942 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=73 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:34:24.955437 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bcc894b-db17-4acf-9972-13c846eeac62 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:34:24.959692 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6448ab60-9443-4ede-9319-90eb0abdbc57 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ed200ce29684c5f80354fa9a8638fe2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0788e6725a84e57a4174284c1845755', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:34:24.960341 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6448ab60-9443-4ede-9319-90eb0abdbc57 service neutron] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:34:24.960627 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6448ab60-9443-4ede-9319-90eb0abdbc57 service neutron] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:34:24.960907 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6448ab60-9443-4ede-9319-90eb0abdbc57 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:34:24.970885 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6448ab60-9443-4ede-9319-90eb0abdbc57 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:34:24.972322 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6448ab60-9443-4ede-9319-90eb0abdbc57 service neutron] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:34:24.982296 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 726/1464] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 1 16:34:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 16:34:27.039702 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37a801c-84e3-497a-a118-a3414a322b3d None None] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:34:27.040238 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37a801c-84e3-497a-a118-a3414a322b3d None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:34:27.040238 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37a801c-84e3-497a-a118-a3414a322b3d None None] PATH_INFO: `/` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:34:27.041667 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 739/1465] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jul 1 16:34:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 16:34:27.047018 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10a3b53-f98a-4e82-b61b-b40c5929546a None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:34:27.047447 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10a3b53-f98a-4e82-b61b-b40c5929546a None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:34:27.047584 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10a3b53-f98a-4e82-b61b-b40c5929546a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:34:27.066086 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-c10a3b53-f98a-4e82-b61b-b40c5929546a None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:34:27.067323 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-c10a3b53-f98a-4e82-b61b-b40c5929546a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:34:27.076493 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 727/1466] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jul 1 16:34:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 16:34:27.114470 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37a801c-84e3-497a-a118-a3414a322b3d None None] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:34:27.125596 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37a801c-84e3-497a-a118-a3414a322b3d None None] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:34:27.129054 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1d1e73-ce6f-4289-b89c-831ad4c0cc90 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ed200ce29684c5f80354fa9a8638fe2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0788e6725a84e57a4174284c1845755', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:34:27.129684 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1d1e73-ce6f-4289-b89c-831ad4c0cc90 service neutron] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:34:27.129966 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1d1e73-ce6f-4289-b89c-831ad4c0cc90 service neutron] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:34:27.130267 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1d1e73-ce6f-4289-b89c-831ad4c0cc90 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:34:27.139777 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb1d1e73-ce6f-4289-b89c-831ad4c0cc90 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:34:27.141201 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb1d1e73-ce6f-4289-b89c-831ad4c0cc90 service neutron] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:34:27.150557 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 740/1467] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 1 16:34:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 16:34:29.209760 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ce4038-c7c3-4a31-8b30-fca4582ddaa7 None None] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:34:29.210282 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ce4038-c7c3-4a31-8b30-fca4582ddaa7 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:34:29.210282 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ce4038-c7c3-4a31-8b30-fca4582ddaa7 None None] PATH_INFO: `/` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:34:29.211667 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 728/1468] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jul 1 16:34:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 16:34:29.217874 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ffa2a1-3405-4b82-8c00-8f3fd2069fdd None None] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:34:29.218293 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ffa2a1-3405-4b82-8c00-8f3fd2069fdd None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:34:29.218496 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ffa2a1-3405-4b82-8c00-8f3fd2069fdd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:34:29.236338 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.auth.core [None req-59ffa2a1-3405-4b82-8c00-8f3fd2069fdd None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80312) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:34:29.237510 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-59ffa2a1-3405-4b82-8c00-8f3fd2069fdd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:34:29.245117 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 741/1469] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jul 1 16:34:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 16:34:29.281762 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ce4038-c7c3-4a31-8b30-fca4582ddaa7 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:34:29.292508 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ce4038-c7c3-4a31-8b30-fca4582ddaa7 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:34:29.296705 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708fe4c3-0251-4564-9bd9-e30d6b866f32 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ed200ce29684c5f80354fa9a8638fe2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0788e6725a84e57a4174284c1845755', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:34:29.297326 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708fe4c3-0251-4564-9bd9-e30d6b866f32 service neutron] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:34:29.297606 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708fe4c3-0251-4564-9bd9-e30d6b866f32 service neutron] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:34:29.297877 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708fe4c3-0251-4564-9bd9-e30d6b866f32 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:34:29.306561 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-708fe4c3-0251-4564-9bd9-e30d6b866f32 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:34:29.307966 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-708fe4c3-0251-4564-9bd9-e30d6b866f32 service neutron] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:34:29.317559 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 729/1470] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 1 16:34:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 01 16:34:39.230785 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=11 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:34:39.313318 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=14 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:34:49.869925 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30a856f-3566-477e-a779-e34f182f5936 None None] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:34:49.870518 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30a856f-3566-477e-a779-e34f182f5936 None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:34:49.870518 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30a856f-3566-477e-a779-e34f182f5936 None None] PATH_INFO: `/` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:34:49.871912 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 742/1471] 10.222.0.57 () {54 vars in 926 bytes} [Fri Jul 1 16:34:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 16:34:49.878235 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4a4492-a000-482e-889f-162f046bb7de None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:34:49.878684 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4a4492-a000-482e-889f-162f046bb7de None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:34:49.878801 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4a4492-a000-482e-889f-162f046bb7de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:34:49.896089 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-ac4a4492-a000-482e-889f-162f046bb7de None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:34:49.897272 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-ac4a4492-a000-482e-889f-162f046bb7de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:34:49.904876 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 730/1472] 10.222.0.57 () {58 vars in 1023 bytes} [Fri Jul 1 16:34:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 16:34:49.952795 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c967273-8c93-4452-8efa-2ccfa21d7530 None None] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:34:49.953113 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c967273-8c93-4452-8efa-2ccfa21d7530 None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:34:49.953399 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c967273-8c93-4452-8efa-2ccfa21d7530 None None] PATH_INFO: `/` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:34:49.954815 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 743/1473] 10.222.0.57 () {54 vars in 962 bytes} [Fri Jul 1 16:34:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 16:34:49.959593 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d696082-3623-4048-868c-0a0262c9bbd4 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:34:49.959885 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d696082-3623-4048-868c-0a0262c9bbd4 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:34:49.960165 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d696082-3623-4048-868c-0a0262c9bbd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:34:49.977938 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-8d696082-3623-4048-868c-0a0262c9bbd4 None None] MFA Rules not processed for user `fd028715bfea43099e7c6cde429ecf92`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:34:49.979139 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-8d696082-3623-4048-868c-0a0262c9bbd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:34:49.986062 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 731/1474] 10.222.0.57 () {58 vars in 1059 bytes} [Fri Jul 1 16:34:49 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 27 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 16:34:49.993336 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c967273-8c93-4452-8efa-2ccfa21d7530 None None] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:34:50.004367 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c967273-8c93-4452-8efa-2ccfa21d7530 None None] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:34:50.007843 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7111cc50-5893-40cc-855d-6c8370a13a78 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd028715bfea43099e7c6cde429ecf92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0788e6725a84e57a4174284c1845755', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:34:50.008477 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7111cc50-5893-40cc-855d-6c8370a13a78 service glance] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:34:50.008764 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7111cc50-5893-40cc-855d-6c8370a13a78 service glance] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:34:50.009043 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7111cc50-5893-40cc-855d-6c8370a13a78 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:34:50.018602 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7111cc50-5893-40cc-855d-6c8370a13a78 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:34:50.020054 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7111cc50-5893-40cc-855d-6c8370a13a78 service glance] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:34:50.029638 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 744/1475] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 1 16:34:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 16:34:52.172316 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8dc64f-675c-4dcd-bdee-9bfaecfe6fe4 None None] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:34:52.172830 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8dc64f-675c-4dcd-bdee-9bfaecfe6fe4 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:34:52.172830 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8dc64f-675c-4dcd-bdee-9bfaecfe6fe4 None None] PATH_INFO: `/` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:34:52.174233 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 732/1476] 10.222.0.57 () {54 vars in 926 bytes} [Fri Jul 1 16:34:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 16:34:52.179538 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c257b8a7-4aa9-4e93-a833-a95fa0d4fc09 None None] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:34:52.179952 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c257b8a7-4aa9-4e93-a833-a95fa0d4fc09 None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:34:52.180106 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c257b8a7-4aa9-4e93-a833-a95fa0d4fc09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:34:52.197729 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.auth.core [None req-c257b8a7-4aa9-4e93-a833-a95fa0d4fc09 None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80312) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:34:52.198961 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-c257b8a7-4aa9-4e93-a833-a95fa0d4fc09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:34:52.207307 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 745/1477] 10.222.0.57 () {58 vars in 1023 bytes} [Fri Jul 1 16:34:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 16:34:52.231543 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8dc64f-675c-4dcd-bdee-9bfaecfe6fe4 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:34:52.244085 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8dc64f-675c-4dcd-bdee-9bfaecfe6fe4 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:34:52.248542 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8164595d-6c7b-4f9a-b2fa-d833b04f5132 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:34:52.249236 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8164595d-6c7b-4f9a-b2fa-d833b04f5132 admin admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:34:52.249627 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8164595d-6c7b-4f9a-b2fa-d833b04f5132 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:34:52.249939 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8164595d-6c7b-4f9a-b2fa-d833b04f5132 admin admin] PATH_INFO: `/v3/projects` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:34:52.250742 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8164595d-6c7b-4f9a-b2fa-d833b04f5132 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:34:52.252502 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: WARNING py.warnings [None req-8164595d-6c7b-4f9a-b2fa-d833b04f5132 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 01 16:34:52.252502 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: warnings.warn(msg) Jul 01 16:34:52.252502 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: Jul 01 16:34:52.253668 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8164595d-6c7b-4f9a-b2fa-d833b04f5132 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:34:52.273064 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 733/1478] 10.222.0.57 () {56 vars in 1108 bytes} [Fri Jul 1 16:34:52 2022] GET /identity/v3/projects => generated 2553 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 16:34:54.388983 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739faba9-23f2-42c8-952a-ac3a2e3737c6 None None] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:34:54.389539 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739faba9-23f2-42c8-952a-ac3a2e3737c6 None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:34:54.389539 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739faba9-23f2-42c8-952a-ac3a2e3737c6 None None] PATH_INFO: `/` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:34:54.391022 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 746/1479] 10.222.0.57 () {54 vars in 926 bytes} [Fri Jul 1 16:34:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 16:34:54.396461 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6260a6c-b8d8-4010-b5ac-af0af8b5dba0 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:34:54.396877 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6260a6c-b8d8-4010-b5ac-af0af8b5dba0 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:34:54.397032 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6260a6c-b8d8-4010-b5ac-af0af8b5dba0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:34:54.415416 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-c6260a6c-b8d8-4010-b5ac-af0af8b5dba0 None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:34:54.416628 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-c6260a6c-b8d8-4010-b5ac-af0af8b5dba0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:34:54.425180 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 734/1480] 10.222.0.57 () {58 vars in 1023 bytes} [Fri Jul 1 16:34:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 01 16:34:54.469394 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739faba9-23f2-42c8-952a-ac3a2e3737c6 None None] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:34:54.484252 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739faba9-23f2-42c8-952a-ac3a2e3737c6 None None] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:34:54.490291 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c94a12-04d2-4512-a31c-aef81dacd55e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a50f745b31941b7a7265c0fc852f48e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0788e6725a84e57a4174284c1845755', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:34:54.490972 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c94a12-04d2-4512-a31c-aef81dacd55e service nova] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:34:54.491256 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c94a12-04d2-4512-a31c-aef81dacd55e service nova] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:34:54.491538 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c94a12-04d2-4512-a31c-aef81dacd55e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:34:54.500862 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c94a12-04d2-4512-a31c-aef81dacd55e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:34:54.502491 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c94a12-04d2-4512-a31c-aef81dacd55e service nova] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:34:54.511810 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 747/1481] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 1 16:34:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 16:34:56.748464 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8f3615-444f-4099-bbdc-1ba3faeca128 None None] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:34:56.748979 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8f3615-444f-4099-bbdc-1ba3faeca128 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:34:56.748979 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8f3615-444f-4099-bbdc-1ba3faeca128 None None] PATH_INFO: `/` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:34:56.751031 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 735/1482] 10.222.0.57 () {54 vars in 926 bytes} [Fri Jul 1 16:34:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 16:34:56.756387 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81164cd8-1f80-4c3e-ab9a-a9a9177d1bfb None None] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:34:56.756848 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81164cd8-1f80-4c3e-ab9a-a9a9177d1bfb None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:34:56.757003 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81164cd8-1f80-4c3e-ab9a-a9a9177d1bfb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:34:56.775287 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.auth.core [None req-81164cd8-1f80-4c3e-ab9a-a9a9177d1bfb None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80312) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:34:56.776468 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-81164cd8-1f80-4c3e-ab9a-a9a9177d1bfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:34:56.785286 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 748/1483] 10.222.0.57 () {58 vars in 1023 bytes} [Fri Jul 1 16:34:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 16:34:56.821250 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8f3615-444f-4099-bbdc-1ba3faeca128 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:34:56.833527 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8f3615-444f-4099-bbdc-1ba3faeca128 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:34:56.837161 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f7c90b-1e04-4599-a0a5-aa06be6291cb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ed200ce29684c5f80354fa9a8638fe2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0788e6725a84e57a4174284c1845755', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:34:56.837803 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f7c90b-1e04-4599-a0a5-aa06be6291cb service neutron] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:34:56.838082 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f7c90b-1e04-4599-a0a5-aa06be6291cb service neutron] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:34:56.838382 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f7c90b-1e04-4599-a0a5-aa06be6291cb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:34:56.849586 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f7c90b-1e04-4599-a0a5-aa06be6291cb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:34:56.850713 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f7c90b-1e04-4599-a0a5-aa06be6291cb service neutron] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:34:56.860395 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 736/1484] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 1 16:34:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 16:34:58.964443 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3ab0d6-35fa-43ec-a882-2868bca97f61 None None] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:34:58.964973 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3ab0d6-35fa-43ec-a882-2868bca97f61 None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:34:58.965091 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3ab0d6-35fa-43ec-a882-2868bca97f61 None None] PATH_INFO: `/` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:34:58.966419 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 749/1485] 10.222.0.57 () {54 vars in 926 bytes} [Fri Jul 1 16:34:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 16:34:58.972519 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2badd85d-e9bb-44a2-bb38-e9ee51d707c6 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:34:58.972940 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2badd85d-e9bb-44a2-bb38-e9ee51d707c6 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:34:58.973086 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2badd85d-e9bb-44a2-bb38-e9ee51d707c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:34:58.990592 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-2badd85d-e9bb-44a2-bb38-e9ee51d707c6 None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:34:58.991719 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-2badd85d-e9bb-44a2-bb38-e9ee51d707c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:34:58.999479 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 737/1486] 10.222.0.57 () {58 vars in 1023 bytes} [Fri Jul 1 16:34:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 16:34:59.038598 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3ab0d6-35fa-43ec-a882-2868bca97f61 None None] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:34:59.049545 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3ab0d6-35fa-43ec-a882-2868bca97f61 None None] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:34:59.053014 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf7ec46-63d3-4721-bc61-a498f53f5587 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ed200ce29684c5f80354fa9a8638fe2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0788e6725a84e57a4174284c1845755', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:34:59.053641 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf7ec46-63d3-4721-bc61-a498f53f5587 service neutron] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:34:59.053922 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf7ec46-63d3-4721-bc61-a498f53f5587 service neutron] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:34:59.054198 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf7ec46-63d3-4721-bc61-a498f53f5587 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:34:59.063785 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf7ec46-63d3-4721-bc61-a498f53f5587 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:34:59.065219 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf7ec46-63d3-4721-bc61-a498f53f5587 service neutron] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:34:59.074917 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 750/1487] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 1 16:34:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 16:35:01.257434 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2385196a-40c5-4f68-852b-3fef2009d530 None None] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:35:01.257997 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2385196a-40c5-4f68-852b-3fef2009d530 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:35:01.257997 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2385196a-40c5-4f68-852b-3fef2009d530 None None] PATH_INFO: `/` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:35:01.259340 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 738/1488] 10.222.0.57 () {54 vars in 926 bytes} [Fri Jul 1 16:35:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 16:35:01.264603 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82182b2e-d725-4061-b93a-982ea86ba0e0 None None] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:35:01.265038 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82182b2e-d725-4061-b93a-982ea86ba0e0 None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:35:01.265666 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82182b2e-d725-4061-b93a-982ea86ba0e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:35:01.284267 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.auth.core [None req-82182b2e-d725-4061-b93a-982ea86ba0e0 None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80312) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:35:01.285468 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-82182b2e-d725-4061-b93a-982ea86ba0e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:35:01.293312 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 751/1489] 10.222.0.57 () {58 vars in 1023 bytes} [Fri Jul 1 16:35:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 16:35:01.334380 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2385196a-40c5-4f68-852b-3fef2009d530 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:35:01.345536 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2385196a-40c5-4f68-852b-3fef2009d530 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:35:01.349201 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0754ac-1422-4ae1-a2ff-c33afcb8643e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ed200ce29684c5f80354fa9a8638fe2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0788e6725a84e57a4174284c1845755', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:35:01.349848 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0754ac-1422-4ae1-a2ff-c33afcb8643e service neutron] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:35:01.350138 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0754ac-1422-4ae1-a2ff-c33afcb8643e service neutron] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:35:01.350452 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0754ac-1422-4ae1-a2ff-c33afcb8643e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:35:01.359949 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc0754ac-1422-4ae1-a2ff-c33afcb8643e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:35:01.361386 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc0754ac-1422-4ae1-a2ff-c33afcb8643e service neutron] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:35:01.371984 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 739/1490] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 1 16:35:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 16:35:03.629062 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70de34ee-d060-4fed-a0a9-4465c837ee70 None None] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:35:03.629631 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70de34ee-d060-4fed-a0a9-4465c837ee70 None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:35:03.629631 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70de34ee-d060-4fed-a0a9-4465c837ee70 None None] PATH_INFO: `/` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:35:03.631039 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 752/1491] 10.222.0.57 () {54 vars in 926 bytes} [Fri Jul 1 16:35:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 16:35:03.636244 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76485d7-c5f4-4d4e-8216-20a49f50289a None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:35:03.636739 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76485d7-c5f4-4d4e-8216-20a49f50289a None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:35:03.636845 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76485d7-c5f4-4d4e-8216-20a49f50289a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:35:03.655739 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-d76485d7-c5f4-4d4e-8216-20a49f50289a None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:35:03.657511 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-d76485d7-c5f4-4d4e-8216-20a49f50289a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:35:03.665255 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 740/1492] 10.222.0.57 () {58 vars in 1023 bytes} [Fri Jul 1 16:35:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 16:35:03.706071 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70de34ee-d060-4fed-a0a9-4465c837ee70 None None] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:35:03.716774 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70de34ee-d060-4fed-a0a9-4465c837ee70 None None] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:35:03.721046 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f60eab-dc17-4ef6-af0a-44e671f446f5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ed200ce29684c5f80354fa9a8638fe2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0788e6725a84e57a4174284c1845755', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:35:03.721685 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f60eab-dc17-4ef6-af0a-44e671f446f5 service neutron] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:35:03.721972 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f60eab-dc17-4ef6-af0a-44e671f446f5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:35:03.722251 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f60eab-dc17-4ef6-af0a-44e671f446f5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:35:03.731089 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f60eab-dc17-4ef6-af0a-44e671f446f5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:35:03.732518 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f60eab-dc17-4ef6-af0a-44e671f446f5 service neutron] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:35:03.742683 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 753/1493] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 1 16:35:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 16:35:13.649773 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=33 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:35:13.738575 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=33 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:36:31.794285 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b444ae-3207-4fe3-8fe9-62ccccf01746 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:36:31.794996 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b444ae-3207-4fe3-8fe9-62ccccf01746 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:36:31.794996 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b444ae-3207-4fe3-8fe9-62ccccf01746 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:36:31.813421 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-e6b444ae-3207-4fe3-8fe9-62ccccf01746 None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:36:31.814645 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-e6b444ae-3207-4fe3-8fe9-62ccccf01746 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:36:31.824593 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 741/1494] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 1 16:36:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 16:36:31.832848 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fcd097e-69c8-4293-894d-34b643850797 None None] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:36:31.833690 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fcd097e-69c8-4293-894d-34b643850797 None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:36:31.833690 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fcd097e-69c8-4293-894d-34b643850797 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:36:31.854067 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.auth.core [None req-3fcd097e-69c8-4293-894d-34b643850797 None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80312) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:36:31.855466 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-3fcd097e-69c8-4293-894d-34b643850797 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:36:31.863224 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 754/1495] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 1 16:36:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 16:36:31.869626 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b444ae-3207-4fe3-8fe9-62ccccf01746 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:36:31.881645 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b444ae-3207-4fe3-8fe9-62ccccf01746 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:36:31.887302 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79d1966-9bfd-4aa1-9fbc-f7618e210d04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:36:31.887963 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79d1966-9bfd-4aa1-9fbc-f7618e210d04 admin admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:36:31.888247 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79d1966-9bfd-4aa1-9fbc-f7618e210d04 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:36:31.888526 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79d1966-9bfd-4aa1-9fbc-f7618e210d04 admin admin] PATH_INFO: `/v3/domains` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:36:31.889255 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79d1966-9bfd-4aa1-9fbc-f7618e210d04 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:36:31.889566 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79d1966-9bfd-4aa1-9fbc-f7618e210d04 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:36:31.890622 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: WARNING py.warnings [None req-e79d1966-9bfd-4aa1-9fbc-f7618e210d04 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 01 16:36:31.890622 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: warnings.warn(msg) Jul 01 16:36:31.890622 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: Jul 01 16:36:31.891525 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79d1966-9bfd-4aa1-9fbc-f7618e210d04 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:36:31.903710 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 742/1496] 10.222.0.57 () {58 vars in 1152 bytes} [Fri Jul 1 16:36:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:36:31.912643 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fcd097e-69c8-4293-894d-34b643850797 None None] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:36:31.922115 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fcd097e-69c8-4293-894d-34b643850797 None None] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:36:31.929038 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a2d195-87b5-417f-96ed-2d494fa6a8b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:36:31.929718 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a2d195-87b5-417f-96ed-2d494fa6a8b8 admin admin] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:36:31.930003 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a2d195-87b5-417f-96ed-2d494fa6a8b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:36:31.930279 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a2d195-87b5-417f-96ed-2d494fa6a8b8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:36:31.931016 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a2d195-87b5-417f-96ed-2d494fa6a8b8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:36:31.935082 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: WARNING py.warnings [None req-10a2d195-87b5-417f-96ed-2d494fa6a8b8 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 01 16:36:31.935082 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: warnings.warn(msg) Jul 01 16:36:31.935082 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: Jul 01 16:36:31.937247 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a2d195-87b5-417f-96ed-2d494fa6a8b8 admin admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:36:31.982433 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 755/1497] 10.222.0.57 () {60 vars in 1152 bytes} [Fri Jul 1 16:36:31 2022] POST /identity/v3/projects => generated 379 bytes in 71 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 01 16:36:31.988043 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79d1966-9bfd-4aa1-9fbc-f7618e210d04 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:36:32.006301 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79d1966-9bfd-4aa1-9fbc-f7618e210d04 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:36:32.074484 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac7bb81-6b73-4872-9b62-723de2769223 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:36:32.075073 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac7bb81-6b73-4872-9b62-723de2769223 admin admin] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:36:32.075357 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac7bb81-6b73-4872-9b62-723de2769223 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:36:32.075634 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac7bb81-6b73-4872-9b62-723de2769223 admin admin] PATH_INFO: `/v3/users` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:36:32.076333 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac7bb81-6b73-4872-9b62-723de2769223 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:36:32.078656 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: WARNING py.warnings [None req-6ac7bb81-6b73-4872-9b62-723de2769223 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 01 16:36:32.078656 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: warnings.warn(msg) Jul 01 16:36:32.078656 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: Jul 01 16:36:32.079867 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac7bb81-6b73-4872-9b62-723de2769223 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:36:32.102367 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 743/1498] 10.222.0.57 () {60 vars in 1143 bytes} [Fri Jul 1 16:36:31 2022] POST /identity/v3/users => generated 290 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 16:36:32.106590 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a2d195-87b5-417f-96ed-2d494fa6a8b8 admin admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:36:32.118958 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a2d195-87b5-417f-96ed-2d494fa6a8b8 admin admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:36:32.124495 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c12e7e0-20a9-45be-b678-4802bddc061c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:36:32.125175 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c12e7e0-20a9-45be-b678-4802bddc061c admin admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:36:32.125463 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c12e7e0-20a9-45be-b678-4802bddc061c admin admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:36:32.125745 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c12e7e0-20a9-45be-b678-4802bddc061c admin admin] PATH_INFO: `/v3/roles` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:36:32.126515 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c12e7e0-20a9-45be-b678-4802bddc061c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:36:32.127535 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: WARNING py.warnings [None req-0c12e7e0-20a9-45be-b678-4802bddc061c admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 01 16:36:32.127535 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: warnings.warn(msg) Jul 01 16:36:32.127535 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: Jul 01 16:36:32.128426 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c12e7e0-20a9-45be-b678-4802bddc061c admin admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:36:32.143323 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 756/1499] 10.222.0.57 () {58 vars in 1121 bytes} [Fri Jul 1 16:36:32 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 16:36:32.148591 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac7bb81-6b73-4872-9b62-723de2769223 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:36:32.158664 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac7bb81-6b73-4872-9b62-723de2769223 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:36:32.163985 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d5e9b0-0179-467e-8aac-7f4938716cec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:36:32.164650 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d5e9b0-0179-467e-8aac-7f4938716cec admin admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:36:32.164935 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d5e9b0-0179-467e-8aac-7f4938716cec admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:36:32.165212 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d5e9b0-0179-467e-8aac-7f4938716cec admin admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:36:32.165947 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d5e9b0-0179-467e-8aac-7f4938716cec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:36:32.167014 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: WARNING py.warnings [None req-73d5e9b0-0179-467e-8aac-7f4938716cec admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 01 16:36:32.167014 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: warnings.warn(msg) Jul 01 16:36:32.167014 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: Jul 01 16:36:32.167899 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d5e9b0-0179-467e-8aac-7f4938716cec admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:36:32.183150 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 744/1500] 10.222.0.57 () {58 vars in 1121 bytes} [Fri Jul 1 16:36:32 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 16:36:32.187428 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c12e7e0-20a9-45be-b678-4802bddc061c admin admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:36:32.198582 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c12e7e0-20a9-45be-b678-4802bddc061c admin admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:36:32.203414 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9a1554-ec8c-4166-9de5-c63fdcab9462 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:36:32.204339 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9a1554-ec8c-4166-9de5-c63fdcab9462 admin admin] REQUEST_METHOD: `PUT` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:36:32.204622 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9a1554-ec8c-4166-9de5-c63fdcab9462 admin admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:36:32.204897 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9a1554-ec8c-4166-9de5-c63fdcab9462 admin admin] PATH_INFO: `/v3/projects/244855ed0b2345e8af717b0fb989b337/users/dc93f81e750d4e27aa91d2c07c39567c/roles/983be40bdab04c478eaa90c6b6a46b14` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:36:32.220246 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf9a1554-ec8c-4166-9de5-c63fdcab9462 admin admin] RBAC: Authorizing `identity:create_grant(project_id=244855ed0b2345e8af717b0fb989b337, user_id=dc93f81e750d4e27aa91d2c07c39567c, role_id=983be40bdab04c478eaa90c6b6a46b14)` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:36:32.221372 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: WARNING py.warnings [None req-cf9a1554-ec8c-4166-9de5-c63fdcab9462 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 01 16:36:32.221372 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: warnings.warn(msg) Jul 01 16:36:32.221372 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: Jul 01 16:36:32.223352 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf9a1554-ec8c-4166-9de5-c63fdcab9462 admin admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:36:32.237292 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 757/1501] 10.222.0.57 () {60 vars in 1482 bytes} [Fri Jul 1 16:36:32 2022] PUT /identity/v3/projects/244855ed0b2345e8af717b0fb989b337/users/dc93f81e750d4e27aa91d2c07c39567c/roles/983be40bdab04c478eaa90c6b6a46b14 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 16:36:32.250016 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e4666e-f436-45cd-9fe7-e0615433c092 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:36:32.250313 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e4666e-f436-45cd-9fe7-e0615433c092 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:36:32.250626 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e4666e-f436-45cd-9fe7-e0615433c092 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:36:32.277240 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-42e4666e-f436-45cd-9fe7-e0615433c092 None None] MFA Rules not processed for user `dc93f81e750d4e27aa91d2c07c39567c`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:36:32.278494 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-42e4666e-f436-45cd-9fe7-e0615433c092 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:36:32.351236 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 745/1502] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 1 16:36:32 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 16:36:32.367955 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6e950b-0f93-46b8-a074-a406853ccbe4 None None] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:36:32.368258 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6e950b-0f93-46b8-a074-a406853ccbe4 None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:36:32.368526 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6e950b-0f93-46b8-a074-a406853ccbe4 None None] PATH_INFO: `/` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:36:32.369907 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 758/1503] 10.222.0.57 () {54 vars in 963 bytes} [Fri Jul 1 16:36:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 16:36:32.375341 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2967903-7b16-42a7-8482-e55525ddbc05 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:36:32.375759 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2967903-7b16-42a7-8482-e55525ddbc05 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:36:32.376042 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2967903-7b16-42a7-8482-e55525ddbc05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:36:32.399141 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-d2967903-7b16-42a7-8482-e55525ddbc05 None None] MFA Rules not processed for user `dbddf971a2354bbdbb9da1d63f3abf88`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:36:32.402114 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-d2967903-7b16-42a7-8482-e55525ddbc05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:36:32.451587 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:36:32.462429 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 746/1504] 10.222.0.57 () {58 vars in 1060 bytes} [Fri Jul 1 16:36:32 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 01 16:36:32.470759 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6e950b-0f93-46b8-a074-a406853ccbe4 None None] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:36:32.482969 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6e950b-0f93-46b8-a074-a406853ccbe4 None None] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:36:32.492536 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197e165d-690c-4be2-9aeb-525b6b196412 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbddf971a2354bbdbb9da1d63f3abf88', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0788e6725a84e57a4174284c1845755', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:36:32.493359 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197e165d-690c-4be2-9aeb-525b6b196412 service cinder] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:36:32.493647 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197e165d-690c-4be2-9aeb-525b6b196412 service cinder] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:36:32.493931 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197e165d-690c-4be2-9aeb-525b6b196412 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:36:32.503656 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-197e165d-690c-4be2-9aeb-525b6b196412 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:36:32.505393 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-197e165d-690c-4be2-9aeb-525b6b196412 service cinder] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:36:32.515988 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 759/1505] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 1 16:36:32 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 16:36:32.560124 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f869533e-b574-4557-9ef3-cf33d3f9a274 None None] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:36:32.560422 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f869533e-b574-4557-9ef3-cf33d3f9a274 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:36:32.560717 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f869533e-b574-4557-9ef3-cf33d3f9a274 None None] PATH_INFO: `/` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:36:32.562038 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 747/1506] 10.222.0.57 () {52 vars in 829 bytes} [Fri Jul 1 16:36:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 16:36:32.566315 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197e165d-690c-4be2-9aeb-525b6b196412 service cinder] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:36:32.635495 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197e165d-690c-4be2-9aeb-525b6b196412 service cinder] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:36:32.640138 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2fb58b5-8b4a-4e7f-8b6f-4c77084fecce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:36:32.641035 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fb58b5-8b4a-4e7f-8b6f-4c77084fecce admin admin] REQUEST_METHOD: `DELETE` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:36:32.641304 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fb58b5-8b4a-4e7f-8b6f-4c77084fecce admin admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:36:32.641582 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fb58b5-8b4a-4e7f-8b6f-4c77084fecce admin admin] PATH_INFO: `/v3/users/dc93f81e750d4e27aa91d2c07c39567c` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:36:32.642825 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2fb58b5-8b4a-4e7f-8b6f-4c77084fecce admin admin] RBAC: Authorizing `identity:delete_user(user_id=dc93f81e750d4e27aa91d2c07c39567c)` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:36:32.643843 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: WARNING py.warnings [None req-d2fb58b5-8b4a-4e7f-8b6f-4c77084fecce admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 01 16:36:32.643843 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: warnings.warn(msg) Jul 01 16:36:32.643843 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: Jul 01 16:36:32.645139 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2fb58b5-8b4a-4e7f-8b6f-4c77084fecce admin admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:36:32.711985 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-d2fb58b5-8b4a-4e7f-8b6f-4c77084fecce admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dc93f81e750d4e27aa91d2c07c39567c'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:36:32.730221 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-d2fb58b5-8b4a-4e7f-8b6f-4c77084fecce admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dc93f81e750d4e27aa91d2c07c39567c'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:36:32.744330 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-d2fb58b5-8b4a-4e7f-8b6f-4c77084fecce admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dc93f81e750d4e27aa91d2c07c39567c'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:36:32.745073 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-d2fb58b5-8b4a-4e7f-8b6f-4c77084fecce admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dc93f81e750d4e27aa91d2c07c39567c'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:36:32.745792 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-d2fb58b5-8b4a-4e7f-8b6f-4c77084fecce admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dc93f81e750d4e27aa91d2c07c39567c'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:36:32.749723 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=88,keystone:INSERT=3,keystone:DELETE=9 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:36:32.762220 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 760/1507] 10.222.0.57 () {58 vars in 1223 bytes} [Fri Jul 1 16:36:32 2022] DELETE /identity/v3/users/dc93f81e750d4e27aa91d2c07c39567c => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 16:36:32.766602 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063edfd0-46d0-42c0-9ab7-8a553b4f55df None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:36:32.766942 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063edfd0-46d0-42c0-9ab7-8a553b4f55df None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:36:32.767223 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063edfd0-46d0-42c0-9ab7-8a553b4f55df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:36:32.785420 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-063edfd0-46d0-42c0-9ab7-8a553b4f55df None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:36:32.786665 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-063edfd0-46d0-42c0-9ab7-8a553b4f55df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:36:32.848074 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 748/1508] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 1 16:36:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 16:36:32.857890 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2fb58b5-8b4a-4e7f-8b6f-4c77084fecce admin admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:36:32.859376 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-d2fb58b5-8b4a-4e7f-8b6f-4c77084fecce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:36:32.917079 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2fb58b5-8b4a-4e7f-8b6f-4c77084fecce admin admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:36:32.920566 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecffb554-f258-49a0-b382-540f091e71f0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ed200ce29684c5f80354fa9a8638fe2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0788e6725a84e57a4174284c1845755', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:36:32.921215 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecffb554-f258-49a0-b382-540f091e71f0 service neutron] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:36:32.921501 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecffb554-f258-49a0-b382-540f091e71f0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:36:32.921780 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecffb554-f258-49a0-b382-540f091e71f0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:36:32.931813 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecffb554-f258-49a0-b382-540f091e71f0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:36:32.933272 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecffb554-f258-49a0-b382-540f091e71f0 service neutron] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:36:32.942837 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 761/1509] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 1 16:36:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 16:36:33.309793 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063edfd0-46d0-42c0-9ab7-8a553b4f55df None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:36:33.311265 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-063edfd0-46d0-42c0-9ab7-8a553b4f55df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:36:33.323165 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063edfd0-46d0-42c0-9ab7-8a553b4f55df None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:36:33.328270 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b24ed76-fcc8-4127-ae55-b81a437c1c5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:36:33.329196 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b24ed76-fcc8-4127-ae55-b81a437c1c5a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:36:33.329420 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b24ed76-fcc8-4127-ae55-b81a437c1c5a admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:36:33.329696 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b24ed76-fcc8-4127-ae55-b81a437c1c5a admin admin] PATH_INFO: `/v3/projects/244855ed0b2345e8af717b0fb989b337` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:36:33.330924 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b24ed76-fcc8-4127-ae55-b81a437c1c5a admin admin] RBAC: Authorizing `identity:delete_project(project_id=244855ed0b2345e8af717b0fb989b337)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:36:33.331960 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: WARNING py.warnings [None req-7b24ed76-fcc8-4127-ae55-b81a437c1c5a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 01 16:36:33.331960 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: warnings.warn(msg) Jul 01 16:36:33.331960 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: Jul 01 16:36:33.333755 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b24ed76-fcc8-4127-ae55-b81a437c1c5a admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:36:33.397849 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-7b24ed76-fcc8-4127-ae55-b81a437c1c5a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '244855ed0b2345e8af717b0fb989b337', 'request_id': 'req-7b24ed76-fcc8-4127-ae55-b81a437c1c5a'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:36:33.403897 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-7b24ed76-fcc8-4127-ae55-b81a437c1c5a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '244855ed0b2345e8af717b0fb989b337', 'request_id': 'req-7b24ed76-fcc8-4127-ae55-b81a437c1c5a'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:36:33.420121 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-7b24ed76-fcc8-4127-ae55-b81a437c1c5a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '244855ed0b2345e8af717b0fb989b337', 'request_id': 'req-7b24ed76-fcc8-4127-ae55-b81a437c1c5a'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:36:33.435058 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-7b24ed76-fcc8-4127-ae55-b81a437c1c5a admin admin] The token cache is being invalidate because project 244855ed0b2345e8af717b0fb989b337 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80311) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 01 16:36:33.435276 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-7b24ed76-fcc8-4127-ae55-b81a437c1c5a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:36:33.435903 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-7b24ed76-fcc8-4127-ae55-b81a437c1c5a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:36:33.437561 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 749/1510] 10.222.0.57 () {58 vars in 1232 bytes} [Fri Jul 1 16:36:33 2022] DELETE /identity/v3/projects/244855ed0b2345e8af717b0fb989b337 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 16:36:42.938638 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:DELETE=1,keystone:SELECT=28 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:36:43.431825 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=55,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:38:15.462624 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38ebf61-a3e6-4c5c-a9e4-bf65e1f797b4 None None] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:38:15.463274 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38ebf61-a3e6-4c5c-a9e4-bf65e1f797b4 None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:38:15.463274 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38ebf61-a3e6-4c5c-a9e4-bf65e1f797b4 None None] PATH_INFO: `/` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:38:15.464662 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 762/1511] 127.0.0.1 () {54 vars in 901 bytes} [Fri Jul 1 16:38:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 16:38:15.469512 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc830ee-cc1c-4e0c-8a7a-349fc3c795a4 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:38:15.469922 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc830ee-cc1c-4e0c-8a7a-349fc3c795a4 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:38:15.470085 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc830ee-cc1c-4e0c-8a7a-349fc3c795a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:38:15.488863 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-dcc830ee-cc1c-4e0c-8a7a-349fc3c795a4 None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:38:15.490131 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-dcc830ee-cc1c-4e0c-8a7a-349fc3c795a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:38:15.551404 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 750/1512] 127.0.0.1 () {58 vars in 998 bytes} [Fri Jul 1 16:38:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 16:38:15.715268 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38ebf61-a3e6-4c5c-a9e4-bf65e1f797b4 None None] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:38:15.716898 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-b38ebf61-a3e6-4c5c-a9e4-bf65e1f797b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:38:15.790294 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38ebf61-a3e6-4c5c-a9e4-bf65e1f797b4 None None] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:38:15.795676 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8595be4-29ed-4c11-9026-8cd2f5bc427a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a50f745b31941b7a7265c0fc852f48e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0788e6725a84e57a4174284c1845755', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:38:15.796321 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8595be4-29ed-4c11-9026-8cd2f5bc427a service nova] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:38:15.796603 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8595be4-29ed-4c11-9026-8cd2f5bc427a service nova] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:38:15.796880 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8595be4-29ed-4c11-9026-8cd2f5bc427a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:38:15.804929 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8595be4-29ed-4c11-9026-8cd2f5bc427a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:38:15.806547 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8595be4-29ed-4c11-9026-8cd2f5bc427a service nova] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:38:15.816116 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 763/1513] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 1 16:38:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 16:38:16.668384 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca5552b-b68b-4841-b077-945882c04469 None None] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:38:16.669033 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca5552b-b68b-4841-b077-945882c04469 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:38:16.669033 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca5552b-b68b-4841-b077-945882c04469 None None] PATH_INFO: `/` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:38:16.670524 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 751/1514] 127.0.0.1 () {54 vars in 901 bytes} [Fri Jul 1 16:38:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 16:38:16.675118 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93123f7b-7d5d-4b06-8d6a-f59cfc62f5dd None None] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:38:16.675555 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93123f7b-7d5d-4b06-8d6a-f59cfc62f5dd None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:38:16.675675 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93123f7b-7d5d-4b06-8d6a-f59cfc62f5dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:38:16.694759 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.auth.core [None req-93123f7b-7d5d-4b06-8d6a-f59cfc62f5dd None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80312) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:38:16.695882 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-93123f7b-7d5d-4b06-8d6a-f59cfc62f5dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:38:16.703826 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 764/1515] 127.0.0.1 () {58 vars in 998 bytes} [Fri Jul 1 16:38:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 16:38:16.861213 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca5552b-b68b-4841-b077-945882c04469 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:38:16.876831 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca5552b-b68b-4841-b077-945882c04469 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:38:16.883286 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff83e782-0f66-4d52-926b-bd6870a78569 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a50f745b31941b7a7265c0fc852f48e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0788e6725a84e57a4174284c1845755', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:38:16.883930 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff83e782-0f66-4d52-926b-bd6870a78569 service nova] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:38:16.884214 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff83e782-0f66-4d52-926b-bd6870a78569 service nova] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:38:16.884497 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff83e782-0f66-4d52-926b-bd6870a78569 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:38:16.892632 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff83e782-0f66-4d52-926b-bd6870a78569 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:38:16.894243 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff83e782-0f66-4d52-926b-bd6870a78569 service nova] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:38:16.904855 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 752/1516] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 1 16:38:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 16:38:17.405536 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2eb4d3-b819-484c-8502-5e1d036dcda5 None None] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:38:17.405825 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2eb4d3-b819-484c-8502-5e1d036dcda5 None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:38:17.406103 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2eb4d3-b819-484c-8502-5e1d036dcda5 None None] PATH_INFO: `/v3` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:38:17.407470 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 765/1517] 10.222.0.43 () {54 vars in 925 bytes} [Fri Jul 1 16:38:17 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:38:17.422229 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4768c2-9779-4f77-8927-19374d9042f4 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:38:17.422552 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4768c2-9779-4f77-8927-19374d9042f4 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:38:17.422841 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4768c2-9779-4f77-8927-19374d9042f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:38:17.441403 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-9b4768c2-9779-4f77-8927-19374d9042f4 None None] MFA Rules not processed for user `bbc38af18dec4e9b8143df23af6aad43`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:38:17.442640 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-9b4768c2-9779-4f77-8927-19374d9042f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:38:17.506655 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 753/1518] 10.222.0.43 () {58 vars in 1015 bytes} [Fri Jul 1 16:38:17 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 16:38:26.428246 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b9239c-8822-4dd6-88ea-10c150403626 None None] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:38:26.428753 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b9239c-8822-4dd6-88ea-10c150403626 None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:38:26.428753 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b9239c-8822-4dd6-88ea-10c150403626 None None] PATH_INFO: `/` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:38:26.430096 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 766/1519] 10.222.0.57 () {54 vars in 960 bytes} [Fri Jul 1 16:38:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 16:38:26.435031 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66155a5b-3ef6-4044-9f17-cdfcc3724a2b None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:38:26.435412 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66155a5b-3ef6-4044-9f17-cdfcc3724a2b None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:38:26.435606 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66155a5b-3ef6-4044-9f17-cdfcc3724a2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:38:26.454928 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-66155a5b-3ef6-4044-9f17-cdfcc3724a2b None None] MFA Rules not processed for user `bbc38af18dec4e9b8143df23af6aad43`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:38:26.456129 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-66155a5b-3ef6-4044-9f17-cdfcc3724a2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:38:26.465945 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 754/1520] 10.222.0.57 () {58 vars in 1057 bytes} [Fri Jul 1 16:38:26 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 16:38:26.473513 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b9239c-8822-4dd6-88ea-10c150403626 None None] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:38:26.487936 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b9239c-8822-4dd6-88ea-10c150403626 None None] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:38:26.494016 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255d21e8-f747-4805-bae2-6f349013ba6e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc38af18dec4e9b8143df23af6aad43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0788e6725a84e57a4174284c1845755', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:38:26.494705 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255d21e8-f747-4805-bae2-6f349013ba6e service placement] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:38:26.494990 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255d21e8-f747-4805-bae2-6f349013ba6e service placement] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:38:26.495268 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255d21e8-f747-4805-bae2-6f349013ba6e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:38:26.511056 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-255d21e8-f747-4805-bae2-6f349013ba6e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:38:26.512799 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-255d21e8-f747-4805-bae2-6f349013ba6e service placement] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:38:26.522849 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 767/1521] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 1 16:38:26 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 16:38:28.152916 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4b84dc-2a5c-4d9c-a03c-23b7405fc6e1 None None] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:38:28.153450 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4b84dc-2a5c-4d9c-a03c-23b7405fc6e1 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:38:28.153450 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4b84dc-2a5c-4d9c-a03c-23b7405fc6e1 None None] PATH_INFO: `/` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:38:28.154862 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 755/1522] 127.0.0.1 () {54 vars in 901 bytes} [Fri Jul 1 16:38:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 16:38:28.159622 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff89c01-87f0-47a7-8355-bd8d30d7cad4 None None] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:38:28.160051 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff89c01-87f0-47a7-8355-bd8d30d7cad4 None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:38:28.160181 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff89c01-87f0-47a7-8355-bd8d30d7cad4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:38:28.178639 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.auth.core [None req-3ff89c01-87f0-47a7-8355-bd8d30d7cad4 None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80312) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:38:28.179848 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-3ff89c01-87f0-47a7-8355-bd8d30d7cad4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:38:28.187595 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 768/1523] 127.0.0.1 () {58 vars in 998 bytes} [Fri Jul 1 16:38:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 16:38:28.353322 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4b84dc-2a5c-4d9c-a03c-23b7405fc6e1 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:38:28.368165 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4b84dc-2a5c-4d9c-a03c-23b7405fc6e1 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:38:28.373550 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ae6be5-9f5c-4d52-b749-b91c66741f27 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a50f745b31941b7a7265c0fc852f48e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0788e6725a84e57a4174284c1845755', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:38:28.374189 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ae6be5-9f5c-4d52-b749-b91c66741f27 service nova] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:38:28.374519 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ae6be5-9f5c-4d52-b749-b91c66741f27 service nova] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:38:28.374804 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ae6be5-9f5c-4d52-b749-b91c66741f27 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:38:28.383757 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ae6be5-9f5c-4d52-b749-b91c66741f27 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:38:28.385355 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ae6be5-9f5c-4d52-b749-b91c66741f27 service nova] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:38:28.394916 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 756/1524] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 1 16:38:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 16:38:29.227043 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e63d98-b851-4cc0-8cf2-a0e5b9812d78 None None] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:38:29.227558 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e63d98-b851-4cc0-8cf2-a0e5b9812d78 None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:38:29.227558 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e63d98-b851-4cc0-8cf2-a0e5b9812d78 None None] PATH_INFO: `/` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:38:29.228945 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 769/1525] 127.0.0.1 () {54 vars in 901 bytes} [Fri Jul 1 16:38:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 16:38:29.233747 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37798fd7-1905-4de5-85d2-7e155224ec24 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:38:29.234155 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37798fd7-1905-4de5-85d2-7e155224ec24 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:38:29.234300 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37798fd7-1905-4de5-85d2-7e155224ec24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:38:29.253114 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-37798fd7-1905-4de5-85d2-7e155224ec24 None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:38:29.254331 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-37798fd7-1905-4de5-85d2-7e155224ec24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:38:29.262834 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 757/1526] 127.0.0.1 () {58 vars in 998 bytes} [Fri Jul 1 16:38:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 01 16:38:29.416883 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e63d98-b851-4cc0-8cf2-a0e5b9812d78 None None] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:38:29.432342 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e63d98-b851-4cc0-8cf2-a0e5b9812d78 None None] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:38:29.437647 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed06e5c1-f3ff-46a8-982b-dc01a996826c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a50f745b31941b7a7265c0fc852f48e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0788e6725a84e57a4174284c1845755', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:38:29.438279 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed06e5c1-f3ff-46a8-982b-dc01a996826c service nova] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:38:29.438591 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed06e5c1-f3ff-46a8-982b-dc01a996826c service nova] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:38:29.438870 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed06e5c1-f3ff-46a8-982b-dc01a996826c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:38:29.447513 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed06e5c1-f3ff-46a8-982b-dc01a996826c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:38:29.449120 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed06e5c1-f3ff-46a8-982b-dc01a996826c service nova] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:38:29.459977 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 770/1527] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 1 16:38:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 16:38:33.639520 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7d8f2c-84ad-420b-b024-af8c97bddfef None None] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:38:33.640194 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7d8f2c-84ad-420b-b024-af8c97bddfef None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:38:33.640194 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7d8f2c-84ad-420b-b024-af8c97bddfef None None] PATH_INFO: `/` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:38:33.641546 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 758/1528] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jul 1 16:38:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 16:38:33.647368 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a6aefa-e039-46af-b237-020a5230b360 None None] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:38:33.647865 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a6aefa-e039-46af-b237-020a5230b360 None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:38:33.647984 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a6aefa-e039-46af-b237-020a5230b360 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:38:33.669456 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.auth.core [None req-a0a6aefa-e039-46af-b237-020a5230b360 None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80312) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:38:33.670681 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-a0a6aefa-e039-46af-b237-020a5230b360 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:38:33.679397 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 771/1529] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jul 1 16:38:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 16:38:33.715945 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7d8f2c-84ad-420b-b024-af8c97bddfef None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:38:33.717664 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-5e7d8f2c-84ad-420b-b024-af8c97bddfef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:38:33.778770 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7d8f2c-84ad-420b-b024-af8c97bddfef None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:38:33.782433 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857e9267-6b5f-4bf3-a5d2-f7119961332c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ed200ce29684c5f80354fa9a8638fe2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0788e6725a84e57a4174284c1845755', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:38:33.783084 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857e9267-6b5f-4bf3-a5d2-f7119961332c service neutron] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:38:33.783373 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857e9267-6b5f-4bf3-a5d2-f7119961332c service neutron] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:38:33.783657 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857e9267-6b5f-4bf3-a5d2-f7119961332c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:38:33.793862 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-857e9267-6b5f-4bf3-a5d2-f7119961332c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:38:33.795337 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-857e9267-6b5f-4bf3-a5d2-f7119961332c service neutron] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:38:33.800286 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:38:33.806677 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 759/1530] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 1 16:38:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 16:38:35.872523 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1d3a69-4b02-41d7-a126-b10fa0fa6fa1 None None] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:38:35.873072 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1d3a69-4b02-41d7-a126-b10fa0fa6fa1 None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:38:35.873072 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1d3a69-4b02-41d7-a126-b10fa0fa6fa1 None None] PATH_INFO: `/` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:38:35.874509 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 772/1531] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jul 1 16:38:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 16:38:35.879615 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b949f1-2e7b-46b1-810d-8a660d5a66d6 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:38:35.880578 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b949f1-2e7b-46b1-810d-8a660d5a66d6 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:38:35.880720 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b949f1-2e7b-46b1-810d-8a660d5a66d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:38:35.899266 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-d4b949f1-2e7b-46b1-810d-8a660d5a66d6 None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:38:35.900485 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-d4b949f1-2e7b-46b1-810d-8a660d5a66d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:38:35.908387 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 760/1532] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jul 1 16:38:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 16:38:35.944377 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1d3a69-4b02-41d7-a126-b10fa0fa6fa1 None None] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:38:35.958120 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1d3a69-4b02-41d7-a126-b10fa0fa6fa1 None None] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:38:35.962585 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffa48d2-06f6-4919-afbe-351b1b9161de service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ed200ce29684c5f80354fa9a8638fe2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0788e6725a84e57a4174284c1845755', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:38:35.963230 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffa48d2-06f6-4919-afbe-351b1b9161de service neutron] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:38:35.963513 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffa48d2-06f6-4919-afbe-351b1b9161de service neutron] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:38:35.963790 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffa48d2-06f6-4919-afbe-351b1b9161de service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:38:35.972800 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dffa48d2-06f6-4919-afbe-351b1b9161de service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:38:35.974428 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dffa48d2-06f6-4919-afbe-351b1b9161de service neutron] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:38:35.983815 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 773/1533] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 1 16:38:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 16:38:39.634618 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a433f5d-7244-449c-affb-ead38f62ab39 None None] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:38:39.635202 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a433f5d-7244-449c-affb-ead38f62ab39 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:38:39.635202 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a433f5d-7244-449c-affb-ead38f62ab39 None None] PATH_INFO: `/` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:38:39.636613 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 761/1534] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jul 1 16:38:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 16:38:39.641794 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc8d4ea-de4c-4458-b598-ab86029f67f7 None None] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:38:39.642199 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc8d4ea-de4c-4458-b598-ab86029f67f7 None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:38:39.642392 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc8d4ea-de4c-4458-b598-ab86029f67f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:38:39.661035 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.auth.core [None req-acc8d4ea-de4c-4458-b598-ab86029f67f7 None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80312) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:38:39.662259 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-acc8d4ea-de4c-4458-b598-ab86029f67f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:38:39.670274 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 774/1535] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jul 1 16:38:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 16:38:39.986886 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a433f5d-7244-449c-affb-ead38f62ab39 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:38:39.988561 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-8a433f5d-7244-449c-affb-ead38f62ab39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:38:40.045839 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a433f5d-7244-449c-affb-ead38f62ab39 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:38:40.049384 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33184684-f49f-4293-b5b0-297266e9e150 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbddf971a2354bbdbb9da1d63f3abf88', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0788e6725a84e57a4174284c1845755', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:38:40.050023 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33184684-f49f-4293-b5b0-297266e9e150 service cinder] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:38:40.050432 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33184684-f49f-4293-b5b0-297266e9e150 service cinder] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:38:40.050663 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33184684-f49f-4293-b5b0-297266e9e150 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:38:40.060355 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33184684-f49f-4293-b5b0-297266e9e150 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:38:40.061778 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33184684-f49f-4293-b5b0-297266e9e150 service cinder] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:38:40.071454 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 762/1536] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 1 16:38:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 16:38:42.135320 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e7268a-c6c3-4342-aadb-5f4599e83a0a None None] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:38:42.135851 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e7268a-c6c3-4342-aadb-5f4599e83a0a None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:38:42.135851 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e7268a-c6c3-4342-aadb-5f4599e83a0a None None] PATH_INFO: `/` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:38:42.137220 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 775/1537] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jul 1 16:38:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 16:38:42.142689 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c07eae-f0e7-4d81-8c6b-9f544bb9e1d0 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:38:42.143107 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c07eae-f0e7-4d81-8c6b-9f544bb9e1d0 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:38:42.143285 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c07eae-f0e7-4d81-8c6b-9f544bb9e1d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:38:42.162093 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-41c07eae-f0e7-4d81-8c6b-9f544bb9e1d0 None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:38:42.163343 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-41c07eae-f0e7-4d81-8c6b-9f544bb9e1d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:38:42.171188 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 763/1538] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jul 1 16:38:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 16:38:42.481910 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e7268a-c6c3-4342-aadb-5f4599e83a0a None None] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:38:42.483625 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-11e7268a-c6c3-4342-aadb-5f4599e83a0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:38:42.496911 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e7268a-c6c3-4342-aadb-5f4599e83a0a None None] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:38:42.500404 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda69261-a827-4ba8-ac90-0865bb183765 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbddf971a2354bbdbb9da1d63f3abf88', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0788e6725a84e57a4174284c1845755', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:38:42.501041 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda69261-a827-4ba8-ac90-0865bb183765 service cinder] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:38:42.501441 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda69261-a827-4ba8-ac90-0865bb183765 service cinder] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:38:42.501726 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda69261-a827-4ba8-ac90-0865bb183765 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:38:42.511272 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda69261-a827-4ba8-ac90-0865bb183765 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:38:42.512699 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda69261-a827-4ba8-ac90-0865bb183765 service cinder] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:38:42.523695 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 776/1539] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 1 16:38:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 16:38:52.156385 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=32 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:38:52.518107 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=70 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:40:12.758893 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4072301a-c23a-4ada-91ff-d01cbff7c343 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:40:12.758893 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4072301a-c23a-4ada-91ff-d01cbff7c343 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:40:12.759697 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4072301a-c23a-4ada-91ff-d01cbff7c343 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:40:12.784030 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-4072301a-c23a-4ada-91ff-d01cbff7c343 None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:40:12.785884 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-4072301a-c23a-4ada-91ff-d01cbff7c343 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:40:12.798648 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 764/1540] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 1 16:40:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 16:40:12.808674 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8fe8b3-f7c6-48b0-b462-e7a6cc2d396d None None] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:40:12.809278 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8fe8b3-f7c6-48b0-b462-e7a6cc2d396d None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:40:12.809278 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8fe8b3-f7c6-48b0-b462-e7a6cc2d396d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:40:12.837284 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.auth.core [None req-fd8fe8b3-f7c6-48b0-b462-e7a6cc2d396d None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80312) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:40:12.838553 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-fd8fe8b3-f7c6-48b0-b462-e7a6cc2d396d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:40:12.862530 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 777/1541] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 1 16:40:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 16:40:12.870636 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4072301a-c23a-4ada-91ff-d01cbff7c343 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:40:12.889921 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4072301a-c23a-4ada-91ff-d01cbff7c343 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:40:12.898863 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21887701-6004-4040-8557-ac3921006902 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:40:12.899618 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21887701-6004-4040-8557-ac3921006902 admin admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:40:12.899908 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21887701-6004-4040-8557-ac3921006902 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:40:12.900186 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21887701-6004-4040-8557-ac3921006902 admin admin] PATH_INFO: `/v3/domains` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:40:12.900983 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21887701-6004-4040-8557-ac3921006902 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:40:12.901321 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21887701-6004-4040-8557-ac3921006902 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:40:12.903034 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21887701-6004-4040-8557-ac3921006902 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:40:12.928561 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 765/1542] 10.222.0.57 () {58 vars in 1152 bytes} [Fri Jul 1 16:40:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:40:12.937008 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8fe8b3-f7c6-48b0-b462-e7a6cc2d396d None None] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:40:12.951379 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8fe8b3-f7c6-48b0-b462-e7a6cc2d396d None None] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:40:12.957005 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c308bb-b8a6-4770-9fe8-6cf0034d207d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:40:12.957778 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c308bb-b8a6-4770-9fe8-6cf0034d207d admin admin] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:40:12.958063 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c308bb-b8a6-4770-9fe8-6cf0034d207d admin admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:40:12.958360 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c308bb-b8a6-4770-9fe8-6cf0034d207d admin admin] PATH_INFO: `/v3/projects` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:40:12.959341 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c308bb-b8a6-4770-9fe8-6cf0034d207d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:40:12.961323 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c308bb-b8a6-4770-9fe8-6cf0034d207d admin admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:40:13.010581 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 778/1543] 10.222.0.57 () {60 vars in 1152 bytes} [Fri Jul 1 16:40:12 2022] POST /identity/v3/projects => generated 383 bytes in 74 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 16:40:13.015214 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21887701-6004-4040-8557-ac3921006902 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:40:13.043259 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21887701-6004-4040-8557-ac3921006902 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:40:13.092859 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4e27b3-2977-44b0-aba5-fb5a6a0ebd82 None None] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:40:13.093224 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4e27b3-2977-44b0-aba5-fb5a6a0ebd82 None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:40:13.093742 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4e27b3-2977-44b0-aba5-fb5a6a0ebd82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:40:13.129330 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.auth.core [None req-8a4e27b3-2977-44b0-aba5-fb5a6a0ebd82 None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80312) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:40:13.131471 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-8a4e27b3-2977-44b0-aba5-fb5a6a0ebd82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:40:13.155906 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6473a72d-2852-49e0-9c20-f394660dc43c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:40:13.156596 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6473a72d-2852-49e0-9c20-f394660dc43c admin admin] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:40:13.156887 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6473a72d-2852-49e0-9c20-f394660dc43c admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:40:13.157171 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6473a72d-2852-49e0-9c20-f394660dc43c admin admin] PATH_INFO: `/v3/users` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:40:13.157866 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6473a72d-2852-49e0-9c20-f394660dc43c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:40:13.160709 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6473a72d-2852-49e0-9c20-f394660dc43c admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:40:13.186782 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 766/1544] 10.222.0.57 () {60 vars in 1143 bytes} [Fri Jul 1 16:40:13 2022] POST /identity/v3/users => generated 292 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 16:40:13.189493 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57e75c6-c928-43d3-94b5-44a94f81b45e None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:40:13.191366 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57e75c6-c928-43d3-94b5-44a94f81b45e None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:40:13.193303 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57e75c6-c928-43d3-94b5-44a94f81b45e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:40:13.223591 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-c57e75c6-c928-43d3-94b5-44a94f81b45e None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:40:13.224884 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-c57e75c6-c928-43d3-94b5-44a94f81b45e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:40:13.228249 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 779/1545] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 1 16:40:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 01 16:40:13.229730 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2339e8-2987-496d-9737-c76cc53a82cf None None] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:40:13.230028 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2339e8-2987-496d-9737-c76cc53a82cf None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:40:13.230312 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2339e8-2987-496d-9737-c76cc53a82cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:40:13.243587 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 767/1546] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 1 16:40:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 16:40:13.245954 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7307bb6b-160c-4ce8-9bfe-5b020d6f73ea None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:40:13.246233 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7307bb6b-160c-4ce8-9bfe-5b020d6f73ea None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:40:13.246544 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7307bb6b-160c-4ce8-9bfe-5b020d6f73ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:40:13.257322 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.auth.core [None req-dc2339e8-2987-496d-9737-c76cc53a82cf None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80312) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:40:13.258992 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-dc2339e8-2987-496d-9737-c76cc53a82cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:40:13.265934 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-7307bb6b-160c-4ce8-9bfe-5b020d6f73ea None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:40:13.266626 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 780/1547] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 1 16:40:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 16:40:13.267166 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-7307bb6b-160c-4ce8-9bfe-5b020d6f73ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:40:13.267420 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2339e8-2987-496d-9737-c76cc53a82cf None None] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:40:13.275431 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 768/1548] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 1 16:40:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 16:40:13.276828 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bcb4366-1abe-4294-9941-2030bbecf65b None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:40:13.277126 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bcb4366-1abe-4294-9941-2030bbecf65b None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:40:13.277411 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bcb4366-1abe-4294-9941-2030bbecf65b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:40:13.279562 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2339e8-2987-496d-9737-c76cc53a82cf None None] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:40:13.284120 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7afc7d01-2ad1-4e5d-bb9f-bb2ad80ad891 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:40:13.285404 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afc7d01-2ad1-4e5d-bb9f-bb2ad80ad891 admin admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:40:13.286252 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afc7d01-2ad1-4e5d-bb9f-bb2ad80ad891 admin admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:40:13.286562 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afc7d01-2ad1-4e5d-bb9f-bb2ad80ad891 admin admin] PATH_INFO: `/v3/roles` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:40:13.287301 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7afc7d01-2ad1-4e5d-bb9f-bb2ad80ad891 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:40:13.288937 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7afc7d01-2ad1-4e5d-bb9f-bb2ad80ad891 admin admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:40:13.296046 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-2bcb4366-1abe-4294-9941-2030bbecf65b None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:40:13.297249 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-2bcb4366-1abe-4294-9941-2030bbecf65b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:40:13.302761 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 781/1549] 10.222.0.57 () {58 vars in 1121 bytes} [Fri Jul 1 16:40:13 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 16:40:13.302761 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1175e2b-dbe9-4bee-a3d2-78cf28270147 None None] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:40:13.302761 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1175e2b-dbe9-4bee-a3d2-78cf28270147 None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:40:13.302761 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1175e2b-dbe9-4bee-a3d2-78cf28270147 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:40:13.306843 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 769/1550] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 1 16:40:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 16:40:13.308250 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2caa39d6-0796-420d-9abe-26e0495dbd19 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:40:13.308613 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2caa39d6-0796-420d-9abe-26e0495dbd19 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:40:13.308973 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2caa39d6-0796-420d-9abe-26e0495dbd19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:40:13.322924 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.auth.core [None req-c1175e2b-dbe9-4bee-a3d2-78cf28270147 None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80312) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:40:13.324156 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-c1175e2b-dbe9-4bee-a3d2-78cf28270147 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:40:13.327669 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-2caa39d6-0796-420d-9abe-26e0495dbd19 None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:40:13.328838 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-2caa39d6-0796-420d-9abe-26e0495dbd19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:40:13.332883 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 782/1551] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 1 16:40:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 16:40:13.334397 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fac3b50-41cb-4d7b-899a-44cadc0cb476 None None] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:40:13.334712 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fac3b50-41cb-4d7b-899a-44cadc0cb476 None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:40:13.334989 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fac3b50-41cb-4d7b-899a-44cadc0cb476 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:40:13.340452 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 770/1552] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 1 16:40:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 16:40:13.341759 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a736fd-b312-42e8-b478-ffec59dde937 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:40:13.342052 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a736fd-b312-42e8-b478-ffec59dde937 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:40:13.342361 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a736fd-b312-42e8-b478-ffec59dde937 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:40:13.360134 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.auth.core [None req-4fac3b50-41cb-4d7b-899a-44cadc0cb476 None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80312) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:40:13.361345 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-d4a736fd-b312-42e8-b478-ffec59dde937 None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:40:13.361492 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-4fac3b50-41cb-4d7b-899a-44cadc0cb476 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:40:13.362526 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-d4a736fd-b312-42e8-b478-ffec59dde937 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:40:13.369889 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 783/1553] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 1 16:40:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 16:40:13.371184 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 771/1554] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 1 16:40:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 16:40:13.371308 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b43c61-164b-4eef-8c92-91a8c1544e6e None None] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:40:13.371399 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b43c61-164b-4eef-8c92-91a8c1544e6e None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:40:13.371679 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b43c61-164b-4eef-8c92-91a8c1544e6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:40:13.372307 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa242d7-141e-4e0a-a31e-8b1fcd3330ef None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:40:13.372594 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa242d7-141e-4e0a-a31e-8b1fcd3330ef None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:40:13.372878 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa242d7-141e-4e0a-a31e-8b1fcd3330ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:40:13.389220 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.auth.core [None req-06b43c61-164b-4eef-8c92-91a8c1544e6e None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80312) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:40:13.390441 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-06b43c61-164b-4eef-8c92-91a8c1544e6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:40:13.394430 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-6fa242d7-141e-4e0a-a31e-8b1fcd3330ef None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:40:13.395763 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-6fa242d7-141e-4e0a-a31e-8b1fcd3330ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:40:13.398868 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 784/1555] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 1 16:40:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 16:40:13.399456 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b43c61-164b-4eef-8c92-91a8c1544e6e None None] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:40:13.407721 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 772/1556] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 1 16:40:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 16:40:13.408578 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b43c61-164b-4eef-8c92-91a8c1544e6e None None] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:40:13.408951 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7949313-610a-498c-844d-f55719f3a5bd None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:40:13.409279 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7949313-610a-498c-844d-f55719f3a5bd None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:40:13.409665 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7949313-610a-498c-844d-f55719f3a5bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:40:13.413746 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe479460-a6c4-46c9-9550-b79ff16557aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:40:13.414432 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe479460-a6c4-46c9-9550-b79ff16557aa admin admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:40:13.414715 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe479460-a6c4-46c9-9550-b79ff16557aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:40:13.414991 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe479460-a6c4-46c9-9550-b79ff16557aa admin admin] PATH_INFO: `/v3/roles` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:40:13.415747 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe479460-a6c4-46c9-9550-b79ff16557aa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:40:13.417351 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe479460-a6c4-46c9-9550-b79ff16557aa admin admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:40:13.427715 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 785/1557] 10.222.0.57 () {58 vars in 1121 bytes} [Fri Jul 1 16:40:13 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 16:40:13.428281 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe479460-a6c4-46c9-9550-b79ff16557aa admin admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:40:13.432622 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-b7949313-610a-498c-844d-f55719f3a5bd None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:40:13.434016 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-b7949313-610a-498c-844d-f55719f3a5bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:40:13.437859 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe479460-a6c4-46c9-9550-b79ff16557aa admin admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:40:13.442803 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05cdb891-11d2-422f-a2fc-edbfcb5cc81b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:40:13.443268 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 773/1558] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 1 16:40:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 16:40:13.443486 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05cdb891-11d2-422f-a2fc-edbfcb5cc81b admin admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:40:13.443730 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05cdb891-11d2-422f-a2fc-edbfcb5cc81b admin admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:40:13.444012 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05cdb891-11d2-422f-a2fc-edbfcb5cc81b admin admin] PATH_INFO: `/v3/domains` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:40:13.444877 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05cdb891-11d2-422f-a2fc-edbfcb5cc81b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80312) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:40:13.445094 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05cdb891-11d2-422f-a2fc-edbfcb5cc81b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:40:13.445198 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7416396-5145-4f97-b246-4655206769b8 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:40:13.445512 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7416396-5145-4f97-b246-4655206769b8 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:40:13.445866 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7416396-5145-4f97-b246-4655206769b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:40:13.446269 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: WARNING py.warnings [None req-05cdb891-11d2-422f-a2fc-edbfcb5cc81b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 01 16:40:13.446269 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: warnings.warn(msg) Jul 01 16:40:13.446269 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: Jul 01 16:40:13.447169 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05cdb891-11d2-422f-a2fc-edbfcb5cc81b admin admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:40:13.460268 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 786/1559] 10.222.0.57 () {58 vars in 1152 bytes} [Fri Jul 1 16:40:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:40:13.461758 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b55fa7-d5bf-46cc-ab4d-b45cf326f2bf None None] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:40:13.462049 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b55fa7-d5bf-46cc-ab4d-b45cf326f2bf None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:40:13.462381 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b55fa7-d5bf-46cc-ab4d-b45cf326f2bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:40:13.465043 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-f7416396-5145-4f97-b246-4655206769b8 None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:40:13.466234 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-f7416396-5145-4f97-b246-4655206769b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:40:13.475399 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 774/1560] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 1 16:40:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 16:40:13.479155 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7416396-5145-4f97-b246-4655206769b8 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:40:13.481427 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.auth.core [None req-74b55fa7-d5bf-46cc-ab4d-b45cf326f2bf None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80312) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:40:13.482861 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-74b55fa7-d5bf-46cc-ab4d-b45cf326f2bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:40:13.489356 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7416396-5145-4f97-b246-4655206769b8 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:40:13.490831 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 787/1561] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 1 16:40:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 16:40:13.491662 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b55fa7-d5bf-46cc-ab4d-b45cf326f2bf None None] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:40:13.497706 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6aaa123-81e9-491a-af09-2389f942eeca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:40:13.498466 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6aaa123-81e9-491a-af09-2389f942eeca admin admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:40:13.498788 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6aaa123-81e9-491a-af09-2389f942eeca admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:40:13.499109 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6aaa123-81e9-491a-af09-2389f942eeca admin admin] PATH_INFO: `/v3/domains` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:40:13.499898 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6aaa123-81e9-491a-af09-2389f942eeca admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:40:13.500207 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6aaa123-81e9-491a-af09-2389f942eeca admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:40:13.501142 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b55fa7-d5bf-46cc-ab4d-b45cf326f2bf None None] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:40:13.502132 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6aaa123-81e9-491a-af09-2389f942eeca admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:40:13.506016 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683c6dbb-eb4a-4c9b-9234-7169f894026e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:40:13.506684 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683c6dbb-eb4a-4c9b-9234-7169f894026e admin admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:40:13.506968 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683c6dbb-eb4a-4c9b-9234-7169f894026e admin admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:40:13.507242 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683c6dbb-eb4a-4c9b-9234-7169f894026e admin admin] PATH_INFO: `/v3/domains` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:40:13.507957 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-683c6dbb-eb4a-4c9b-9234-7169f894026e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80312) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:40:13.508496 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-683c6dbb-eb4a-4c9b-9234-7169f894026e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:40:13.510277 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-683c6dbb-eb4a-4c9b-9234-7169f894026e admin admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:40:13.514281 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 775/1562] 10.222.0.57 () {58 vars in 1152 bytes} [Fri Jul 1 16:40:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:40:13.515125 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6aaa123-81e9-491a-af09-2389f942eeca admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:40:13.527548 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6aaa123-81e9-491a-af09-2389f942eeca admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:40:13.528646 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 788/1563] 10.222.0.57 () {58 vars in 1152 bytes} [Fri Jul 1 16:40:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:40:13.529276 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683c6dbb-eb4a-4c9b-9234-7169f894026e admin admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:40:13.532481 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a47b5a-6c7a-46ea-ad35-d39c8cec5d2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:40:13.533127 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a47b5a-6c7a-46ea-ad35-d39c8cec5d2c admin admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:40:13.533410 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a47b5a-6c7a-46ea-ad35-d39c8cec5d2c admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:40:13.533689 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a47b5a-6c7a-46ea-ad35-d39c8cec5d2c admin admin] PATH_INFO: `/v3/domains` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:40:13.534436 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a47b5a-6c7a-46ea-ad35-d39c8cec5d2c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:40:13.535566 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a47b5a-6c7a-46ea-ad35-d39c8cec5d2c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:40:13.537188 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a47b5a-6c7a-46ea-ad35-d39c8cec5d2c admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:40:13.541168 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683c6dbb-eb4a-4c9b-9234-7169f894026e admin admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:40:13.546360 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb85ad6-2146-475c-85e8-033615b810d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:40:13.547411 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb85ad6-2146-475c-85e8-033615b810d6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:40:13.547695 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb85ad6-2146-475c-85e8-033615b810d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:40:13.547975 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb85ad6-2146-475c-85e8-033615b810d6 admin admin] PATH_INFO: `/v3/projects/5747a83d930f4075a78e9b14abd58c7f/users/7c4da21fbfa3472cac6e766a2f4799d8/roles/983be40bdab04c478eaa90c6b6a46b14` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:40:13.550327 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 776/1564] 10.222.0.57 () {58 vars in 1152 bytes} [Fri Jul 1 16:40:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:40:13.550982 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a47b5a-6c7a-46ea-ad35-d39c8cec5d2c admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:40:13.560803 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a47b5a-6c7a-46ea-ad35-d39c8cec5d2c admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:40:13.565759 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b189ee6a-a0e3-4aca-b6e5-2ef13fa6917d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:40:13.566440 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b189ee6a-a0e3-4aca-b6e5-2ef13fa6917d admin admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:40:13.566726 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b189ee6a-a0e3-4aca-b6e5-2ef13fa6917d admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:40:13.567002 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b189ee6a-a0e3-4aca-b6e5-2ef13fa6917d admin admin] PATH_INFO: `/v3/domains` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:40:13.567756 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b189ee6a-a0e3-4aca-b6e5-2ef13fa6917d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:40:13.568065 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b189ee6a-a0e3-4aca-b6e5-2ef13fa6917d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:40:13.569645 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b189ee6a-a0e3-4aca-b6e5-2ef13fa6917d admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:40:13.570564 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb85ad6-2146-475c-85e8-033615b810d6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5747a83d930f4075a78e9b14abd58c7f, user_id=7c4da21fbfa3472cac6e766a2f4799d8, role_id=983be40bdab04c478eaa90c6b6a46b14)` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:40:13.573235 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb85ad6-2146-475c-85e8-033615b810d6 admin admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:40:13.582868 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 777/1565] 10.222.0.57 () {58 vars in 1152 bytes} [Fri Jul 1 16:40:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:40:13.584285 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b189ee6a-a0e3-4aca-b6e5-2ef13fa6917d admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:40:13.594281 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b189ee6a-a0e3-4aca-b6e5-2ef13fa6917d admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:40:13.595074 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 789/1566] 10.222.0.57 () {60 vars in 1482 bytes} [Fri Jul 1 16:40:13 2022] PUT /identity/v3/projects/5747a83d930f4075a78e9b14abd58c7f/users/7c4da21fbfa3472cac6e766a2f4799d8/roles/983be40bdab04c478eaa90c6b6a46b14 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 16:40:13.595893 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb85ad6-2146-475c-85e8-033615b810d6 admin admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:40:13.628737 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:40:13.635351 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:40:13.663877 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fce6af4-2fcc-4da3-9877-3b8ef260d2d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:40:13.664572 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fce6af4-2fcc-4da3-9877-3b8ef260d2d2 admin admin] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:40:13.664860 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fce6af4-2fcc-4da3-9877-3b8ef260d2d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:40:13.664984 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb85ad6-2146-475c-85e8-033615b810d6 admin admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:40:13.665140 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fce6af4-2fcc-4da3-9877-3b8ef260d2d2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:40:13.665845 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fce6af4-2fcc-4da3-9877-3b8ef260d2d2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:40:13.666898 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: WARNING py.warnings [None req-6fce6af4-2fcc-4da3-9877-3b8ef260d2d2 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 01 16:40:13.666898 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: warnings.warn(msg) Jul 01 16:40:13.666898 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: Jul 01 16:40:13.668106 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fce6af4-2fcc-4da3-9877-3b8ef260d2d2 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:40:13.669622 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7ec0e7-d83b-41e7-940d-b05ca7431fcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:40:13.670285 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7ec0e7-d83b-41e7-940d-b05ca7431fcd admin admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:40:13.670598 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7ec0e7-d83b-41e7-940d-b05ca7431fcd admin admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:40:13.670882 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7ec0e7-d83b-41e7-940d-b05ca7431fcd admin admin] PATH_INFO: `/v3/domains` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:40:13.671619 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b7ec0e7-d83b-41e7-940d-b05ca7431fcd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80312) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:40:13.671933 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b7ec0e7-d83b-41e7-940d-b05ca7431fcd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:40:13.673540 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b7ec0e7-d83b-41e7-940d-b05ca7431fcd admin admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:40:13.685775 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 790/1567] 10.222.0.57 () {58 vars in 1152 bytes} [Fri Jul 1 16:40:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:40:13.686740 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7ec0e7-d83b-41e7-940d-b05ca7431fcd admin admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:40:13.697004 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7ec0e7-d83b-41e7-940d-b05ca7431fcd admin admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:40:13.701404 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 778/1568] 10.222.0.57 () {60 vars in 1152 bytes} [Fri Jul 1 16:40:13 2022] POST /identity/v3/projects => generated 405 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 16:40:13.701740 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7b0b5d-a88d-4b47-b5cd-44636e4d5379 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:40:13.702414 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7b0b5d-a88d-4b47-b5cd-44636e4d5379 admin admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:40:13.702523 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fce6af4-2fcc-4da3-9877-3b8ef260d2d2 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:40:13.702711 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7b0b5d-a88d-4b47-b5cd-44636e4d5379 admin admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:40:13.702986 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7b0b5d-a88d-4b47-b5cd-44636e4d5379 admin admin] PATH_INFO: `/v3/domains` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:40:13.703711 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a7b0b5d-a88d-4b47-b5cd-44636e4d5379 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80312) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:40:13.704039 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a7b0b5d-a88d-4b47-b5cd-44636e4d5379 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:40:13.705952 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a7b0b5d-a88d-4b47-b5cd-44636e4d5379 admin admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:40:13.726687 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 791/1569] 10.222.0.57 () {58 vars in 1152 bytes} [Fri Jul 1 16:40:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:40:13.726687 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7b0b5d-a88d-4b47-b5cd-44636e4d5379 admin admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:40:13.736449 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7b0b5d-a88d-4b47-b5cd-44636e4d5379 admin admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:40:13.771211 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fce6af4-2fcc-4da3-9877-3b8ef260d2d2 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:40:13.775991 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c783a3-76a3-4f41-81fa-d6d441403c73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:40:13.776658 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c783a3-76a3-4f41-81fa-d6d441403c73 admin admin] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:40:13.776943 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c783a3-76a3-4f41-81fa-d6d441403c73 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:40:13.777230 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c783a3-76a3-4f41-81fa-d6d441403c73 admin admin] PATH_INFO: `/v3/projects` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:40:13.778007 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c783a3-76a3-4f41-81fa-d6d441403c73 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:40:13.779945 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c783a3-76a3-4f41-81fa-d6d441403c73 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:40:13.793498 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5cec412-4503-4d84-b168-736ab683987a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:40:13.794162 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cec412-4503-4d84-b168-736ab683987a admin admin] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:40:13.794482 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cec412-4503-4d84-b168-736ab683987a admin admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:40:13.794766 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cec412-4503-4d84-b168-736ab683987a admin admin] PATH_INFO: `/v3/projects` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:40:13.795519 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5cec412-4503-4d84-b168-736ab683987a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:40:13.797595 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5cec412-4503-4d84-b168-736ab683987a admin admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:40:13.805684 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 779/1570] 10.222.0.57 () {60 vars in 1152 bytes} [Fri Jul 1 16:40:13 2022] POST /identity/v3/projects => generated 385 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 16:40:13.809586 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c783a3-76a3-4f41-81fa-d6d441403c73 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:40:13.823824 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 792/1571] 10.222.0.57 () {60 vars in 1152 bytes} [Fri Jul 1 16:40:13 2022] POST /identity/v3/projects => generated 379 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 16:40:13.824742 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5cec412-4503-4d84-b168-736ab683987a admin admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:40:13.893964 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5cec412-4503-4d84-b168-736ab683987a admin admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:40:13.898856 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873ca0cb-0083-42a8-b28b-94e71958a92b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:40:13.899547 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873ca0cb-0083-42a8-b28b-94e71958a92b admin admin] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:40:13.899835 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873ca0cb-0083-42a8-b28b-94e71958a92b admin admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:40:13.900147 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873ca0cb-0083-42a8-b28b-94e71958a92b admin admin] PATH_INFO: `/v3/projects` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:40:13.900867 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873ca0cb-0083-42a8-b28b-94e71958a92b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:40:13.902876 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873ca0cb-0083-42a8-b28b-94e71958a92b admin admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:40:13.936467 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 793/1572] 10.222.0.57 () {60 vars in 1152 bytes} [Fri Jul 1 16:40:13 2022] POST /identity/v3/projects => generated 385 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 16:40:13.937720 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18dbe34-f33d-442c-b447-6fc3bdebe6c4 None None] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:40:13.938018 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18dbe34-f33d-442c-b447-6fc3bdebe6c4 None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:40:13.938302 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18dbe34-f33d-442c-b447-6fc3bdebe6c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:40:13.955908 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.auth.core [None req-e18dbe34-f33d-442c-b447-6fc3bdebe6c4 None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80312) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:40:13.957213 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-e18dbe34-f33d-442c-b447-6fc3bdebe6c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:40:13.969516 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:40:13.996732 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c783a3-76a3-4f41-81fa-d6d441403c73 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:40:14.021542 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 794/1573] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 1 16:40:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 16:40:14.022730 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18dbe34-f33d-442c-b447-6fc3bdebe6c4 None None] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:40:14.032150 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18dbe34-f33d-442c-b447-6fc3bdebe6c4 None None] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:40:14.037987 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6274d652-06ff-49ad-b858-31bf7fcef859 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:40:14.038674 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6274d652-06ff-49ad-b858-31bf7fcef859 admin admin] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:40:14.038961 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6274d652-06ff-49ad-b858-31bf7fcef859 admin admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:40:14.039239 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6274d652-06ff-49ad-b858-31bf7fcef859 admin admin] PATH_INFO: `/v3/projects` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:40:14.039932 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6274d652-06ff-49ad-b858-31bf7fcef859 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:40:14.041836 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6274d652-06ff-49ad-b858-31bf7fcef859 admin admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:40:14.047272 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:40:14.056282 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93b4871-ee50-4fd5-9b63-32097c90f702 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:40:14.057218 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93b4871-ee50-4fd5-9b63-32097c90f702 admin admin] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:40:14.057614 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93b4871-ee50-4fd5-9b63-32097c90f702 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:40:14.058452 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93b4871-ee50-4fd5-9b63-32097c90f702 admin admin] PATH_INFO: `/v3/projects` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:40:14.059357 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a93b4871-ee50-4fd5-9b63-32097c90f702 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:40:14.061376 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a93b4871-ee50-4fd5-9b63-32097c90f702 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:40:14.067902 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 795/1574] 10.222.0.57 () {60 vars in 1152 bytes} [Fri Jul 1 16:40:14 2022] POST /identity/v3/projects => generated 373 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 16:40:14.068763 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6274d652-06ff-49ad-b858-31bf7fcef859 admin admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:40:14.089551 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 780/1575] 10.222.0.57 () {60 vars in 1152 bytes} [Fri Jul 1 16:40:13 2022] POST /identity/v3/projects => generated 393 bytes in 283 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 16:40:14.090424 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93b4871-ee50-4fd5-9b63-32097c90f702 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:40:14.136152 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6274d652-06ff-49ad-b858-31bf7fcef859 admin admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:40:14.155251 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93b4871-ee50-4fd5-9b63-32097c90f702 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:40:14.159924 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9f0bef-f632-423c-8dc6-7cff94ead637 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:40:14.160576 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9f0bef-f632-423c-8dc6-7cff94ead637 admin admin] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:40:14.160985 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9f0bef-f632-423c-8dc6-7cff94ead637 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:40:14.161261 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9f0bef-f632-423c-8dc6-7cff94ead637 admin admin] PATH_INFO: `/v3/projects` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:40:14.161978 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e9f0bef-f632-423c-8dc6-7cff94ead637 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:40:14.163963 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e9f0bef-f632-423c-8dc6-7cff94ead637 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:40:14.190532 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 781/1576] 10.222.0.57 () {60 vars in 1152 bytes} [Fri Jul 1 16:40:14 2022] POST /identity/v3/projects => generated 389 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 16:40:14.191730 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9f0bef-f632-423c-8dc6-7cff94ead637 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:40:14.194753 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7deef0e-55ce-4da4-a8db-398687220654 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:40:14.195424 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7deef0e-55ce-4da4-a8db-398687220654 admin admin] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:40:14.195710 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7deef0e-55ce-4da4-a8db-398687220654 admin admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:40:14.196012 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7deef0e-55ce-4da4-a8db-398687220654 admin admin] PATH_INFO: `/v3/users` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:40:14.196702 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7deef0e-55ce-4da4-a8db-398687220654 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:40:14.197715 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: WARNING py.warnings [None req-d7deef0e-55ce-4da4-a8db-398687220654 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 01 16:40:14.197715 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: warnings.warn(msg) Jul 01 16:40:14.197715 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: Jul 01 16:40:14.199013 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7deef0e-55ce-4da4-a8db-398687220654 admin admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:40:14.220140 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 796/1577] 10.222.0.57 () {60 vars in 1143 bytes} [Fri Jul 1 16:40:14 2022] POST /identity/v3/users => generated 303 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 16:40:14.221011 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7deef0e-55ce-4da4-a8db-398687220654 admin admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:40:14.231625 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7deef0e-55ce-4da4-a8db-398687220654 admin admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:40:14.259840 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9f0bef-f632-423c-8dc6-7cff94ead637 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:40:14.263048 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:40:14.264883 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1caa207-084d-43cc-a8ee-c83ba30ccd6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:40:14.265553 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1caa207-084d-43cc-a8ee-c83ba30ccd6c admin admin] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:40:14.265847 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1caa207-084d-43cc-a8ee-c83ba30ccd6c admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:40:14.266129 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1caa207-084d-43cc-a8ee-c83ba30ccd6c admin admin] PATH_INFO: `/v3/users` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:40:14.266862 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1caa207-084d-43cc-a8ee-c83ba30ccd6c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:40:14.269423 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1caa207-084d-43cc-a8ee-c83ba30ccd6c admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:40:14.292257 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 782/1578] 10.222.0.57 () {60 vars in 1143 bytes} [Fri Jul 1 16:40:14 2022] POST /identity/v3/users => generated 293 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 16:40:14.292996 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1caa207-084d-43cc-a8ee-c83ba30ccd6c admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:40:14.293581 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c183a5-b29e-4625-a4b2-4c480d31c6d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:40:14.294324 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c183a5-b29e-4625-a4b2-4c480d31c6d3 admin admin] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:40:14.294701 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c183a5-b29e-4625-a4b2-4c480d31c6d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:40:14.295339 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c183a5-b29e-4625-a4b2-4c480d31c6d3 admin admin] PATH_INFO: `/v3/users` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:40:14.295993 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c183a5-b29e-4625-a4b2-4c480d31c6d3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:40:14.297914 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c183a5-b29e-4625-a4b2-4c480d31c6d3 admin admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:40:14.303314 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1caa207-084d-43cc-a8ee-c83ba30ccd6c admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:40:14.310110 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7235dc21-b906-44da-bb02-4a2bbea4b13e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:40:14.310923 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7235dc21-b906-44da-bb02-4a2bbea4b13e admin admin] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:40:14.311209 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7235dc21-b906-44da-bb02-4a2bbea4b13e admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:40:14.311526 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7235dc21-b906-44da-bb02-4a2bbea4b13e admin admin] PATH_INFO: `/v3/users` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:40:14.312288 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7235dc21-b906-44da-bb02-4a2bbea4b13e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:40:14.314787 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7235dc21-b906-44da-bb02-4a2bbea4b13e admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:40:14.320512 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 797/1579] 10.222.0.57 () {60 vars in 1143 bytes} [Fri Jul 1 16:40:14 2022] POST /identity/v3/users => generated 290 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 16:40:14.321456 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c183a5-b29e-4625-a4b2-4c480d31c6d3 admin admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:40:14.323317 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-19c183a5-b29e-4625-a4b2-4c480d31c6d3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:40:14.340904 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 783/1580] 10.222.0.57 () {60 vars in 1143 bytes} [Fri Jul 1 16:40:14 2022] POST /identity/v3/users => generated 293 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 16:40:14.341638 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7235dc21-b906-44da-bb02-4a2bbea4b13e admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:40:14.352510 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7235dc21-b906-44da-bb02-4a2bbea4b13e admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:40:14.357666 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a249b9c-08af-475f-952d-ca9a2a8c3613 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:40:14.358922 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a249b9c-08af-475f-952d-ca9a2a8c3613 admin admin] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:40:14.359209 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a249b9c-08af-475f-952d-ca9a2a8c3613 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:40:14.359491 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a249b9c-08af-475f-952d-ca9a2a8c3613 admin admin] PATH_INFO: `/v3/users` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:40:14.360185 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a249b9c-08af-475f-952d-ca9a2a8c3613 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:40:14.362087 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a249b9c-08af-475f-952d-ca9a2a8c3613 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:40:14.382787 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 784/1581] 10.222.0.57 () {60 vars in 1143 bytes} [Fri Jul 1 16:40:14 2022] POST /identity/v3/users => generated 287 bytes in 41 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 16:40:14.383608 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a249b9c-08af-475f-952d-ca9a2a8c3613 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:40:14.388727 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c183a5-b29e-4625-a4b2-4c480d31c6d3 admin admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:40:14.392540 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3f4937-95b6-4b09-966a-75152cbf9ed7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ed200ce29684c5f80354fa9a8638fe2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0788e6725a84e57a4174284c1845755', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:40:14.392921 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a249b9c-08af-475f-952d-ca9a2a8c3613 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:40:14.393186 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3f4937-95b6-4b09-966a-75152cbf9ed7 service neutron] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:40:14.393471 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3f4937-95b6-4b09-966a-75152cbf9ed7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:40:14.393750 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3f4937-95b6-4b09-966a-75152cbf9ed7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:40:14.397950 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c89910-8d37-41c3-9f4c-cf800d9d1d59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:40:14.398629 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c89910-8d37-41c3-9f4c-cf800d9d1d59 admin admin] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:40:14.398908 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c89910-8d37-41c3-9f4c-cf800d9d1d59 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:40:14.399210 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c89910-8d37-41c3-9f4c-cf800d9d1d59 admin admin] PATH_INFO: `/v3/users` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:40:14.399893 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c89910-8d37-41c3-9f4c-cf800d9d1d59 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:40:14.401803 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c89910-8d37-41c3-9f4c-cf800d9d1d59 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:40:14.403832 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce3f4937-95b6-4b09-966a-75152cbf9ed7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:40:14.405261 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce3f4937-95b6-4b09-966a-75152cbf9ed7 service neutron] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:40:14.415303 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 798/1582] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 1 16:40:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 16:40:14.416005 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3f4937-95b6-4b09-966a-75152cbf9ed7 service neutron] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:40:14.425898 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 785/1583] 10.222.0.57 () {60 vars in 1143 bytes} [Fri Jul 1 16:40:14 2022] POST /identity/v3/users => generated 297 bytes in 39 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 16:40:14.425898 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c89910-8d37-41c3-9f4c-cf800d9d1d59 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:40:14.428287 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3f4937-95b6-4b09-966a-75152cbf9ed7 service neutron] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:40:14.429916 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=86,keystone:INSERT=14 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:40:14.434888 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f708fa3-70dc-4cee-94cc-2db686cf2db3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:40:14.435584 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f708fa3-70dc-4cee-94cc-2db686cf2db3 admin admin] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:40:14.435849 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c89910-8d37-41c3-9f4c-cf800d9d1d59 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:40:14.435981 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f708fa3-70dc-4cee-94cc-2db686cf2db3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:40:14.436139 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f708fa3-70dc-4cee-94cc-2db686cf2db3 admin admin] PATH_INFO: `/v3/users` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:40:14.436869 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f708fa3-70dc-4cee-94cc-2db686cf2db3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:40:14.439005 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f708fa3-70dc-4cee-94cc-2db686cf2db3 admin admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:40:14.443323 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa78886-adca-496c-9df9-f0733bfdd272 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:40:14.444141 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa78886-adca-496c-9df9-f0733bfdd272 admin admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:40:14.444582 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa78886-adca-496c-9df9-f0733bfdd272 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:40:14.444860 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa78886-adca-496c-9df9-f0733bfdd272 admin admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:40:14.445662 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa78886-adca-496c-9df9-f0733bfdd272 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:40:14.447479 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa78886-adca-496c-9df9-f0733bfdd272 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:40:14.460297 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 786/1584] 10.222.0.57 () {58 vars in 1121 bytes} [Fri Jul 1 16:40:14 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 16:40:14.461466 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa78886-adca-496c-9df9-f0733bfdd272 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:40:14.461962 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 799/1585] 10.222.0.57 () {60 vars in 1143 bytes} [Fri Jul 1 16:40:14 2022] POST /identity/v3/users => generated 295 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 16:40:14.462866 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f708fa3-70dc-4cee-94cc-2db686cf2db3 admin admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:40:14.473310 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa78886-adca-496c-9df9-f0733bfdd272 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:40:14.476073 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f708fa3-70dc-4cee-94cc-2db686cf2db3 admin admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:40:14.478040 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64fed213-1d91-4e70-bc19-efa24394a29e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:40:14.478800 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fed213-1d91-4e70-bc19-efa24394a29e admin admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:40:14.479148 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fed213-1d91-4e70-bc19-efa24394a29e admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:40:14.479492 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fed213-1d91-4e70-bc19-efa24394a29e admin admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:40:14.480223 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64fed213-1d91-4e70-bc19-efa24394a29e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:40:14.480986 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90dcab78-1b40-4c89-8fab-f02e5ff2f1fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:40:14.481726 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90dcab78-1b40-4c89-8fab-f02e5ff2f1fc admin admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:40:14.481983 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64fed213-1d91-4e70-bc19-efa24394a29e admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:40:14.482084 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90dcab78-1b40-4c89-8fab-f02e5ff2f1fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:40:14.482475 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90dcab78-1b40-4c89-8fab-f02e5ff2f1fc admin admin] PATH_INFO: `/v3/roles` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:40:14.483293 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90dcab78-1b40-4c89-8fab-f02e5ff2f1fc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:40:14.484964 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90dcab78-1b40-4c89-8fab-f02e5ff2f1fc admin admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:40:14.494012 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 787/1586] 10.222.0.57 () {58 vars in 1121 bytes} [Fri Jul 1 16:40:14 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 16:40:14.494898 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64fed213-1d91-4e70-bc19-efa24394a29e admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:40:14.499060 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 800/1587] 10.222.0.57 () {58 vars in 1121 bytes} [Fri Jul 1 16:40:14 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 16:40:14.499060 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90dcab78-1b40-4c89-8fab-f02e5ff2f1fc admin admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:40:14.505949 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64fed213-1d91-4e70-bc19-efa24394a29e admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:40:14.511806 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617b953b-70c6-429d-9d1b-7072fff96032 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:40:14.512515 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617b953b-70c6-429d-9d1b-7072fff96032 admin admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:40:14.512825 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617b953b-70c6-429d-9d1b-7072fff96032 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:40:14.513103 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617b953b-70c6-429d-9d1b-7072fff96032 admin admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:40:14.513838 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-617b953b-70c6-429d-9d1b-7072fff96032 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:40:14.515525 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-617b953b-70c6-429d-9d1b-7072fff96032 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:40:14.516515 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90dcab78-1b40-4c89-8fab-f02e5ff2f1fc admin admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:40:14.522854 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9613ccf0-93ef-4c8e-9a6c-b9d50bb05e78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:40:14.523454 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9613ccf0-93ef-4c8e-9a6c-b9d50bb05e78 admin admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:40:14.523731 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9613ccf0-93ef-4c8e-9a6c-b9d50bb05e78 admin admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:40:14.524007 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9613ccf0-93ef-4c8e-9a6c-b9d50bb05e78 admin admin] PATH_INFO: `/v3/roles` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:40:14.524732 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9613ccf0-93ef-4c8e-9a6c-b9d50bb05e78 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:40:14.526334 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9613ccf0-93ef-4c8e-9a6c-b9d50bb05e78 admin admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:40:14.527950 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 788/1588] 10.222.0.57 () {58 vars in 1121 bytes} [Fri Jul 1 16:40:14 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 16:40:14.529064 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617b953b-70c6-429d-9d1b-7072fff96032 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:40:14.540772 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617b953b-70c6-429d-9d1b-7072fff96032 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:40:14.543182 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 801/1589] 10.222.0.57 () {58 vars in 1121 bytes} [Fri Jul 1 16:40:14 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 16:40:14.543923 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9613ccf0-93ef-4c8e-9a6c-b9d50bb05e78 admin admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:40:14.549324 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf977d0-9d48-4da1-8832-14176399826b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:40:14.550089 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf977d0-9d48-4da1-8832-14176399826b admin admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:40:14.551061 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf977d0-9d48-4da1-8832-14176399826b admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:40:14.551347 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf977d0-9d48-4da1-8832-14176399826b admin admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:40:14.552115 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf977d0-9d48-4da1-8832-14176399826b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:40:14.553735 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf977d0-9d48-4da1-8832-14176399826b admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:40:14.555761 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9613ccf0-93ef-4c8e-9a6c-b9d50bb05e78 admin admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:40:14.561420 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44864c11-300a-488f-ab58-82cd896f4c2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:40:14.562114 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44864c11-300a-488f-ab58-82cd896f4c2e admin admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:40:14.562423 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44864c11-300a-488f-ab58-82cd896f4c2e admin admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:40:14.562701 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44864c11-300a-488f-ab58-82cd896f4c2e admin admin] PATH_INFO: `/v3/roles` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:40:14.563428 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44864c11-300a-488f-ab58-82cd896f4c2e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:40:14.565038 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 789/1590] 10.222.0.57 () {58 vars in 1121 bytes} [Fri Jul 1 16:40:14 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 16:40:14.565313 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44864c11-300a-488f-ab58-82cd896f4c2e admin admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:40:14.566131 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf977d0-9d48-4da1-8832-14176399826b admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:40:14.576736 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf977d0-9d48-4da1-8832-14176399826b admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:40:14.585399 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 802/1591] 10.222.0.57 () {58 vars in 1121 bytes} [Fri Jul 1 16:40:14 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 16:40:14.585399 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44864c11-300a-488f-ab58-82cd896f4c2e admin admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:40:14.596640 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44864c11-300a-488f-ab58-82cd896f4c2e admin admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:40:14.596851 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ae5337-61e1-4712-a366-68b11eeb6075 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:40:14.599012 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ae5337-61e1-4712-a366-68b11eeb6075 admin admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:40:14.599637 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ae5337-61e1-4712-a366-68b11eeb6075 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:40:14.600012 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ae5337-61e1-4712-a366-68b11eeb6075 admin admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:40:14.600774 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ae5337-61e1-4712-a366-68b11eeb6075 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:40:14.602213 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7a976f-bf39-496e-bdde-556521007ed0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:40:14.602470 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ae5337-61e1-4712-a366-68b11eeb6075 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:40:14.602935 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7a976f-bf39-496e-bdde-556521007ed0 admin admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:40:14.603224 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7a976f-bf39-496e-bdde-556521007ed0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:40:14.603508 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7a976f-bf39-496e-bdde-556521007ed0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:40:14.604243 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae7a976f-bf39-496e-bdde-556521007ed0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:40:14.605885 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae7a976f-bf39-496e-bdde-556521007ed0 admin admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:40:14.614567 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 790/1592] 10.222.0.57 () {58 vars in 1121 bytes} [Fri Jul 1 16:40:14 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 16:40:14.615441 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ae5337-61e1-4712-a366-68b11eeb6075 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:40:14.620911 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 803/1593] 10.222.0.57 () {58 vars in 1121 bytes} [Fri Jul 1 16:40:14 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 16:40:14.622105 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7a976f-bf39-496e-bdde-556521007ed0 admin admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:40:14.625509 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ae5337-61e1-4712-a366-68b11eeb6075 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:40:14.630574 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ae2541-200f-41ca-b283-6ffb146fe7d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:40:14.631227 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ae2541-200f-41ca-b283-6ffb146fe7d7 admin admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:40:14.631516 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ae2541-200f-41ca-b283-6ffb146fe7d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:40:14.631797 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ae2541-200f-41ca-b283-6ffb146fe7d7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:40:14.632535 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ae2541-200f-41ca-b283-6ffb146fe7d7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:40:14.633214 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7a976f-bf39-496e-bdde-556521007ed0 admin admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:40:14.634267 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ae2541-200f-41ca-b283-6ffb146fe7d7 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:40:14.638482 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b5b24a-2855-47c4-b2f3-ad3525055e6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:40:14.639296 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b5b24a-2855-47c4-b2f3-ad3525055e6f admin admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:40:14.639720 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b5b24a-2855-47c4-b2f3-ad3525055e6f admin admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:40:14.640009 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b5b24a-2855-47c4-b2f3-ad3525055e6f admin admin] PATH_INFO: `/v3/roles` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:40:14.641197 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b5b24a-2855-47c4-b2f3-ad3525055e6f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:40:14.642947 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b5b24a-2855-47c4-b2f3-ad3525055e6f admin admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:40:14.648562 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 791/1594] 10.222.0.57 () {58 vars in 1121 bytes} [Fri Jul 1 16:40:14 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 16:40:14.649548 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ae2541-200f-41ca-b283-6ffb146fe7d7 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:40:14.658426 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 804/1595] 10.222.0.57 () {58 vars in 1121 bytes} [Fri Jul 1 16:40:14 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 16:40:14.660284 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b5b24a-2855-47c4-b2f3-ad3525055e6f admin admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:40:14.668217 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ae2541-200f-41ca-b283-6ffb146fe7d7 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:40:14.673843 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b5b24a-2855-47c4-b2f3-ad3525055e6f admin admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:40:14.674000 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d588528a-0614-4fdd-9405-4a46e64afa1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:40:14.674995 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d588528a-0614-4fdd-9405-4a46e64afa1b admin admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:40:14.676299 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d588528a-0614-4fdd-9405-4a46e64afa1b admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:40:14.676969 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d588528a-0614-4fdd-9405-4a46e64afa1b admin admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:40:14.677991 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d588528a-0614-4fdd-9405-4a46e64afa1b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:40:14.679205 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a1699e-35d5-4eb7-aa08-4abd29f84136 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:40:14.679865 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a1699e-35d5-4eb7-aa08-4abd29f84136 admin admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:40:14.680176 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a1699e-35d5-4eb7-aa08-4abd29f84136 admin admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:40:14.680450 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a1699e-35d5-4eb7-aa08-4abd29f84136 admin admin] PATH_INFO: `/v3/roles` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:40:14.681184 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a1699e-35d5-4eb7-aa08-4abd29f84136 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:40:14.681522 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d588528a-0614-4fdd-9405-4a46e64afa1b admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:40:14.682802 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a1699e-35d5-4eb7-aa08-4abd29f84136 admin admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:40:14.693825 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 805/1596] 10.222.0.57 () {58 vars in 1121 bytes} [Fri Jul 1 16:40:14 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 16:40:14.694879 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a1699e-35d5-4eb7-aa08-4abd29f84136 admin admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:40:14.695326 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 792/1597] 10.222.0.57 () {58 vars in 1121 bytes} [Fri Jul 1 16:40:14 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 16:40:14.696282 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d588528a-0614-4fdd-9405-4a46e64afa1b admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:40:14.710591 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a1699e-35d5-4eb7-aa08-4abd29f84136 admin admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:40:14.711951 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d588528a-0614-4fdd-9405-4a46e64afa1b admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:40:14.715479 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad36479-c32f-4d21-864f-ed3c83f8729d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:40:14.716441 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad36479-c32f-4d21-864f-ed3c83f8729d admin admin] REQUEST_METHOD: `PUT` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:40:14.716794 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad36479-c32f-4d21-864f-ed3c83f8729d admin admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:40:14.717139 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad36479-c32f-4d21-864f-ed3c83f8729d admin admin] PATH_INFO: `/v3/projects/8e11ec92d7d24814be64e3dc5f1ae0a4/users/f5ce7822f6b2469cba93544928f7957f/roles/983be40bdab04c478eaa90c6b6a46b14` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:40:14.717302 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea12fb8-8305-49ba-adaa-76c03b069cb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:40:14.717994 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea12fb8-8305-49ba-adaa-76c03b069cb4 admin admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:40:14.718508 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea12fb8-8305-49ba-adaa-76c03b069cb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:40:14.718799 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea12fb8-8305-49ba-adaa-76c03b069cb4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:40:14.719540 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea12fb8-8305-49ba-adaa-76c03b069cb4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:40:14.721718 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea12fb8-8305-49ba-adaa-76c03b069cb4 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:40:14.733338 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad36479-c32f-4d21-864f-ed3c83f8729d admin admin] RBAC: Authorizing `identity:create_grant(project_id=8e11ec92d7d24814be64e3dc5f1ae0a4, user_id=f5ce7822f6b2469cba93544928f7957f, role_id=983be40bdab04c478eaa90c6b6a46b14)` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:40:14.737442 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 793/1598] 10.222.0.57 () {58 vars in 1121 bytes} [Fri Jul 1 16:40:14 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 16:40:14.737442 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea12fb8-8305-49ba-adaa-76c03b069cb4 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:40:14.737818 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad36479-c32f-4d21-864f-ed3c83f8729d admin admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:40:14.748964 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea12fb8-8305-49ba-adaa-76c03b069cb4 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:40:14.752381 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 806/1599] 10.222.0.57 () {60 vars in 1482 bytes} [Fri Jul 1 16:40:14 2022] PUT /identity/v3/projects/8e11ec92d7d24814be64e3dc5f1ae0a4/users/f5ce7822f6b2469cba93544928f7957f/roles/983be40bdab04c478eaa90c6b6a46b14 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 16:40:14.753202 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad36479-c32f-4d21-864f-ed3c83f8729d admin admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:40:14.779778 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:40:14.824276 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b795923-6d39-456f-a02d-f952ef063274 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:40:14.825853 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b795923-6d39-456f-a02d-f952ef063274 admin admin] REQUEST_METHOD: `PUT` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:40:14.826506 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b795923-6d39-456f-a02d-f952ef063274 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:40:14.826996 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b795923-6d39-456f-a02d-f952ef063274 admin admin] PATH_INFO: `/v3/projects/eec1d42b049c4f8080d138d248280cac/users/aca6b4b731ed4347827748e05fb284ea/roles/983be40bdab04c478eaa90c6b6a46b14` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:40:14.829877 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad36479-c32f-4d21-864f-ed3c83f8729d admin admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:40:14.841726 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473f6deb-35b6-45fb-9ec4-addadd0e83e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:40:14.846316 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473f6deb-35b6-45fb-9ec4-addadd0e83e1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:40:14.846316 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473f6deb-35b6-45fb-9ec4-addadd0e83e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:40:14.846316 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473f6deb-35b6-45fb-9ec4-addadd0e83e1 admin admin] PATH_INFO: `/v3/projects/2c49e143df7c4d059430b7dc5bffd475/users/ea27a708e2c84ccc905ae783524bd12f/roles/983be40bdab04c478eaa90c6b6a46b14` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:40:14.851707 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b795923-6d39-456f-a02d-f952ef063274 admin admin] RBAC: Authorizing `identity:create_grant(project_id=eec1d42b049c4f8080d138d248280cac, user_id=aca6b4b731ed4347827748e05fb284ea, role_id=983be40bdab04c478eaa90c6b6a46b14)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:40:14.852870 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: WARNING py.warnings [None req-9b795923-6d39-456f-a02d-f952ef063274 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 01 16:40:14.852870 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: warnings.warn(msg) Jul 01 16:40:14.852870 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: Jul 01 16:40:14.855246 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b795923-6d39-456f-a02d-f952ef063274 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:40:14.866122 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473f6deb-35b6-45fb-9ec4-addadd0e83e1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2c49e143df7c4d059430b7dc5bffd475, user_id=ea27a708e2c84ccc905ae783524bd12f, role_id=983be40bdab04c478eaa90c6b6a46b14)` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:40:14.870100 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473f6deb-35b6-45fb-9ec4-addadd0e83e1 admin admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:40:14.872214 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 794/1600] 10.222.0.57 () {60 vars in 1482 bytes} [Fri Jul 1 16:40:14 2022] PUT /identity/v3/projects/eec1d42b049c4f8080d138d248280cac/users/aca6b4b731ed4347827748e05fb284ea/roles/983be40bdab04c478eaa90c6b6a46b14 => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 16:40:14.873325 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b795923-6d39-456f-a02d-f952ef063274 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:40:14.888790 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 807/1601] 10.222.0.57 () {60 vars in 1482 bytes} [Fri Jul 1 16:40:14 2022] PUT /identity/v3/projects/2c49e143df7c4d059430b7dc5bffd475/users/ea27a708e2c84ccc905ae783524bd12f/roles/983be40bdab04c478eaa90c6b6a46b14 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 16:40:14.889823 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473f6deb-35b6-45fb-9ec4-addadd0e83e1 admin admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:40:14.946211 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:40:14.950593 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b795923-6d39-456f-a02d-f952ef063274 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:40:14.968011 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473f6deb-35b6-45fb-9ec4-addadd0e83e1 admin admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:40:14.976055 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0263678-f499-470e-9119-a4203a99e717 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:40:14.977732 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0263678-f499-470e-9119-a4203a99e717 admin admin] REQUEST_METHOD: `PUT` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:40:14.978108 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0263678-f499-470e-9119-a4203a99e717 admin admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:40:14.978551 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0263678-f499-470e-9119-a4203a99e717 admin admin] PATH_INFO: `/v3/projects/fe48aa4179354b4fac35b7415ea47f28/users/95ee9f7b04e240d28946d6cb8c24a8d8/roles/983be40bdab04c478eaa90c6b6a46b14` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:40:15.001116 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0263678-f499-470e-9119-a4203a99e717 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fe48aa4179354b4fac35b7415ea47f28, user_id=95ee9f7b04e240d28946d6cb8c24a8d8, role_id=983be40bdab04c478eaa90c6b6a46b14)` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:40:15.006652 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0263678-f499-470e-9119-a4203a99e717 admin admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:40:15.028090 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 808/1602] 10.222.0.57 () {60 vars in 1482 bytes} [Fri Jul 1 16:40:14 2022] PUT /identity/v3/projects/fe48aa4179354b4fac35b7415ea47f28/users/95ee9f7b04e240d28946d6cb8c24a8d8/roles/983be40bdab04c478eaa90c6b6a46b14 => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 16:40:15.029121 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0263678-f499-470e-9119-a4203a99e717 admin admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:40:15.041549 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a390f1-56b5-4fc2-b63f-3d506b8dad33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:40:15.042748 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a390f1-56b5-4fc2-b63f-3d506b8dad33 admin admin] REQUEST_METHOD: `PUT` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:40:15.043074 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a390f1-56b5-4fc2-b63f-3d506b8dad33 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:40:15.043434 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a390f1-56b5-4fc2-b63f-3d506b8dad33 admin admin] PATH_INFO: `/v3/projects/315dd9cfab134d008039e1229e42a96e/users/e471b31763074167ba81c2c25f39b857/roles/983be40bdab04c478eaa90c6b6a46b14` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:40:15.064662 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a390f1-56b5-4fc2-b63f-3d506b8dad33 admin admin] RBAC: Authorizing `identity:create_grant(project_id=315dd9cfab134d008039e1229e42a96e, user_id=e471b31763074167ba81c2c25f39b857, role_id=983be40bdab04c478eaa90c6b6a46b14)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:40:15.068211 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a390f1-56b5-4fc2-b63f-3d506b8dad33 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:40:15.083534 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 795/1603] 10.222.0.57 () {60 vars in 1482 bytes} [Fri Jul 1 16:40:14 2022] PUT /identity/v3/projects/315dd9cfab134d008039e1229e42a96e/users/e471b31763074167ba81c2c25f39b857/roles/983be40bdab04c478eaa90c6b6a46b14 => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 16:40:15.084544 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a390f1-56b5-4fc2-b63f-3d506b8dad33 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:40:15.108150 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0263678-f499-470e-9119-a4203a99e717 admin admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:40:15.153594 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:40:15.172566 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a390f1-56b5-4fc2-b63f-3d506b8dad33 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:40:15.178655 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c431734-f74b-42e3-afca-c25aa84f3525 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:40:15.180278 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c431734-f74b-42e3-afca-c25aa84f3525 admin admin] REQUEST_METHOD: `PUT` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:40:15.180278 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c431734-f74b-42e3-afca-c25aa84f3525 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:40:15.180278 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c431734-f74b-42e3-afca-c25aa84f3525 admin admin] PATH_INFO: `/v3/projects/1fabfb2a97fa4894901bd3d193b3387d/users/dd8e8f465ad14b7c9e1340bfe18d4ff2/roles/983be40bdab04c478eaa90c6b6a46b14` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:40:15.193429 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb63eadc-f85b-4c72-b0c9-dd03becf45f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:40:15.194782 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb63eadc-f85b-4c72-b0c9-dd03becf45f3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:40:15.195415 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb63eadc-f85b-4c72-b0c9-dd03becf45f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:40:15.195617 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb63eadc-f85b-4c72-b0c9-dd03becf45f3 admin admin] PATH_INFO: `/v3/projects/bf55d220c94d4445af9f73c532029e42/users/71c383c04f9543d8ab97a6c0db268ca5/roles/983be40bdab04c478eaa90c6b6a46b14` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:40:15.198140 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c431734-f74b-42e3-afca-c25aa84f3525 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1fabfb2a97fa4894901bd3d193b3387d, user_id=dd8e8f465ad14b7c9e1340bfe18d4ff2, role_id=983be40bdab04c478eaa90c6b6a46b14)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:40:15.200919 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c431734-f74b-42e3-afca-c25aa84f3525 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:40:15.216002 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb63eadc-f85b-4c72-b0c9-dd03becf45f3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bf55d220c94d4445af9f73c532029e42, user_id=71c383c04f9543d8ab97a6c0db268ca5, role_id=983be40bdab04c478eaa90c6b6a46b14)` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:40:15.219262 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb63eadc-f85b-4c72-b0c9-dd03becf45f3 admin admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:40:15.220990 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 796/1604] 10.222.0.57 () {60 vars in 1482 bytes} [Fri Jul 1 16:40:15 2022] PUT /identity/v3/projects/1fabfb2a97fa4894901bd3d193b3387d/users/dd8e8f465ad14b7c9e1340bfe18d4ff2/roles/983be40bdab04c478eaa90c6b6a46b14 => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 16:40:15.224297 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698531f4-6901-4d72-837d-888e9dfea21a None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:40:15.224959 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698531f4-6901-4d72-837d-888e9dfea21a None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:40:15.225227 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698531f4-6901-4d72-837d-888e9dfea21a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:40:15.246015 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-698531f4-6901-4d72-837d-888e9dfea21a None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:40:15.248127 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-698531f4-6901-4d72-837d-888e9dfea21a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:40:15.252329 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 809/1605] 10.222.0.57 () {60 vars in 1482 bytes} [Fri Jul 1 16:40:15 2022] PUT /identity/v3/projects/bf55d220c94d4445af9f73c532029e42/users/71c383c04f9543d8ab97a6c0db268ca5/roles/983be40bdab04c478eaa90c6b6a46b14 => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 16:40:15.252329 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe78d84-8163-47e1-a4c6-edc3265c3964 None None] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:40:15.252329 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe78d84-8163-47e1-a4c6-edc3265c3964 None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:40:15.252329 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe78d84-8163-47e1-a4c6-edc3265c3964 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:40:15.288376 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.auth.core [None req-abe78d84-8163-47e1-a4c6-edc3265c3964 None None] MFA Rules not processed for user `aca6b4b731ed4347827748e05fb284ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80312) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:40:15.289775 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-abe78d84-8163-47e1-a4c6-edc3265c3964 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:40:15.341486 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 797/1606] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 1 16:40:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 16:40:15.342814 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009689c1-932c-43e7-b675-1fdcef278936 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:40:15.343109 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009689c1-932c-43e7-b675-1fdcef278936 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:40:15.349941 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009689c1-932c-43e7-b675-1fdcef278936 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:40:15.372415 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-009689c1-932c-43e7-b675-1fdcef278936 None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:40:15.373960 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-009689c1-932c-43e7-b675-1fdcef278936 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:40:15.386096 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 798/1607] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 1 16:40:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 16:40:15.387464 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0357fc6f-3526-4490-aa01-7ac881a280c2 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:40:15.387741 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0357fc6f-3526-4490-aa01-7ac881a280c2 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:40:15.388020 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0357fc6f-3526-4490-aa01-7ac881a280c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:40:15.391886 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 810/1608] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 1 16:40:15 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 16:40:15.393392 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46913106-9d8c-4d89-8349-5726dc0f6f7c None None] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:40:15.393692 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46913106-9d8c-4d89-8349-5726dc0f6f7c None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:40:15.394269 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46913106-9d8c-4d89-8349-5726dc0f6f7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:40:15.412752 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:40:15.425240 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.auth.core [None req-46913106-9d8c-4d89-8349-5726dc0f6f7c None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80312) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:40:15.427453 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-46913106-9d8c-4d89-8349-5726dc0f6f7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:40:15.439429 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 811/1609] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 1 16:40:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 16:40:15.442013 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3ba487-502a-4e21-aa24-a21a0868fea1 None None] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:40:15.442369 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3ba487-502a-4e21-aa24-a21a0868fea1 None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:40:15.442610 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3ba487-502a-4e21-aa24-a21a0868fea1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:40:15.459304 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-0357fc6f-3526-4490-aa01-7ac881a280c2 None None] MFA Rules not processed for user `95ee9f7b04e240d28946d6cb8c24a8d8`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:40:15.460512 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-0357fc6f-3526-4490-aa01-7ac881a280c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:40:15.485522 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.auth.core [None req-4e3ba487-502a-4e21-aa24-a21a0868fea1 None None] MFA Rules not processed for user `dd8e8f465ad14b7c9e1340bfe18d4ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=80312) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:40:15.489632 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-4e3ba487-502a-4e21-aa24-a21a0868fea1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:40:15.581215 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 799/1610] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 1 16:40:15 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 16:40:15.585277 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ba000b-4c92-4f38-bd85-2257b223802c None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:40:15.585713 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ba000b-4c92-4f38-bd85-2257b223802c None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:40:15.586171 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ba000b-4c92-4f38-bd85-2257b223802c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:40:15.595921 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 812/1611] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 1 16:40:15 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 16:40:15.597657 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3ba487-502a-4e21-aa24-a21a0868fea1 None None] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:40:15.612271 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-19ba000b-4c92-4f38-bd85-2257b223802c None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:40:15.615253 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-19ba000b-4c92-4f38-bd85-2257b223802c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:40:15.638668 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 800/1612] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 1 16:40:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 16:40:15.638668 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ba000b-4c92-4f38-bd85-2257b223802c None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:40:15.647253 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:40:15.687440 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3ba487-502a-4e21-aa24-a21a0868fea1 None None] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:40:15.691600 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835fbbc2-f190-40b7-838e-5c98d58c2cb9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ed200ce29684c5f80354fa9a8638fe2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0788e6725a84e57a4174284c1845755', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:40:15.692506 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835fbbc2-f190-40b7-838e-5c98d58c2cb9 service neutron] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:40:15.692888 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835fbbc2-f190-40b7-838e-5c98d58c2cb9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:40:15.693260 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835fbbc2-f190-40b7-838e-5c98d58c2cb9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:40:15.708309 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-835fbbc2-f190-40b7-838e-5c98d58c2cb9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:40:15.710510 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-835fbbc2-f190-40b7-838e-5c98d58c2cb9 service neutron] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:40:15.711928 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ba000b-4c92-4f38-bd85-2257b223802c None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:40:15.718115 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd13be6a-5fae-4b35-8786-846ce0d31ed2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ed200ce29684c5f80354fa9a8638fe2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0788e6725a84e57a4174284c1845755', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:40:15.719691 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd13be6a-5fae-4b35-8786-846ce0d31ed2 service neutron] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:40:15.719984 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd13be6a-5fae-4b35-8786-846ce0d31ed2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:40:15.720267 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd13be6a-5fae-4b35-8786-846ce0d31ed2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:40:15.724080 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 813/1613] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 1 16:40:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 16:40:15.724989 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835fbbc2-f190-40b7-838e-5c98d58c2cb9 service neutron] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:40:15.733807 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd13be6a-5fae-4b35-8786-846ce0d31ed2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:40:15.735965 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835fbbc2-f190-40b7-838e-5c98d58c2cb9 service neutron] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:40:15.736414 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd13be6a-5fae-4b35-8786-846ce0d31ed2 service neutron] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:40:15.740686 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4873a8-6744-423e-9019-83cd7ce55f48 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ed200ce29684c5f80354fa9a8638fe2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0788e6725a84e57a4174284c1845755', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:40:15.741360 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4873a8-6744-423e-9019-83cd7ce55f48 service neutron] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:40:15.741886 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4873a8-6744-423e-9019-83cd7ce55f48 service neutron] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:40:15.741886 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4873a8-6744-423e-9019-83cd7ce55f48 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:40:15.752449 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4873a8-6744-423e-9019-83cd7ce55f48 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:40:15.754000 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4873a8-6744-423e-9019-83cd7ce55f48 service neutron] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:40:15.754611 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 801/1614] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 1 16:40:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 16:40:15.756391 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0cdbb1-2b89-4ff2-aec9-a09b91070c51 None None] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:40:15.756391 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0cdbb1-2b89-4ff2-aec9-a09b91070c51 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:40:15.756689 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0cdbb1-2b89-4ff2-aec9-a09b91070c51 None None] PATH_INFO: `/` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:40:15.763947 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 802/1615] 10.222.0.57 () {54 vars in 961 bytes} [Fri Jul 1 16:40:15 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 16:40:15.765251 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0cdbb1-2b89-4ff2-aec9-a09b91070c51 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:40:15.766711 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 814/1616] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 1 16:40:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 16:40:15.767606 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4873a8-6744-423e-9019-83cd7ce55f48 service neutron] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:40:15.780499 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4873a8-6744-423e-9019-83cd7ce55f48 service neutron] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:40:15.782634 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0cdbb1-2b89-4ff2-aec9-a09b91070c51 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:40:15.787245 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dae938d-e7f8-4e01-8076-4566615faae5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:40:15.788088 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dae938d-e7f8-4e01-8076-4566615faae5 admin admin] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:40:15.788731 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dae938d-e7f8-4e01-8076-4566615faae5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:40:15.789012 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dae938d-e7f8-4e01-8076-4566615faae5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:40:15.789591 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4ccc35-e3c3-4ac1-be86-033a628e22de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:40:15.789979 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dae938d-e7f8-4e01-8076-4566615faae5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:40:15.790271 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4ccc35-e3c3-4ac1-be86-033a628e22de admin admin] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:40:15.790600 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4ccc35-e3c3-4ac1-be86-033a628e22de admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:40:15.790884 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4ccc35-e3c3-4ac1-be86-033a628e22de admin admin] PATH_INFO: `/v3/projects` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:40:15.791585 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c4ccc35-e3c3-4ac1-be86-033a628e22de admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:40:15.792116 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dae938d-e7f8-4e01-8076-4566615faae5 admin admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:40:15.794531 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c4ccc35-e3c3-4ac1-be86-033a628e22de admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:40:15.833254 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 815/1617] 10.222.0.57 () {60 vars in 1152 bytes} [Fri Jul 1 16:40:15 2022] POST /identity/v3/projects => generated 389 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 16:40:15.834965 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dae938d-e7f8-4e01-8076-4566615faae5 admin admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:40:15.854565 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 803/1618] 10.222.0.57 () {60 vars in 1152 bytes} [Fri Jul 1 16:40:15 2022] POST /identity/v3/projects => generated 371 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 16:40:15.857010 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7922d57a-e4ad-41f5-870b-307ef12f2c10 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:40:15.857526 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7922d57a-e4ad-41f5-870b-307ef12f2c10 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:40:15.857875 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7922d57a-e4ad-41f5-870b-307ef12f2c10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:40:15.882398 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-7922d57a-e4ad-41f5-870b-307ef12f2c10 None None] MFA Rules not processed for user `7a50f745b31941b7a7265c0fc852f48e`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:40:15.884462 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-7922d57a-e4ad-41f5-870b-307ef12f2c10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:40:15.938033 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dae938d-e7f8-4e01-8076-4566615faae5 admin admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:40:15.987145 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:40:16.009348 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 804/1619] 10.222.0.57 () {58 vars in 1058 bytes} [Fri Jul 1 16:40:15 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 16:40:16.012096 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0116945-7111-49be-aa2e-0c7056873e4a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ed200ce29684c5f80354fa9a8638fe2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0788e6725a84e57a4174284c1845755', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:40:16.012899 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0116945-7111-49be-aa2e-0c7056873e4a service neutron] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:40:16.013187 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0116945-7111-49be-aa2e-0c7056873e4a service neutron] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:40:16.013463 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0116945-7111-49be-aa2e-0c7056873e4a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:40:16.018439 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7922d57a-e4ad-41f5-870b-307ef12f2c10 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:40:16.104598 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7922d57a-e4ad-41f5-870b-307ef12f2c10 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:40:16.109914 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696ce0b8-00b1-4f89-abcf-2ff77634df50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:40:16.110781 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696ce0b8-00b1-4f89-abcf-2ff77634df50 admin admin] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:40:16.111155 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696ce0b8-00b1-4f89-abcf-2ff77634df50 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:40:16.111505 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696ce0b8-00b1-4f89-abcf-2ff77634df50 admin admin] PATH_INFO: `/v3/users` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:40:16.112281 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-696ce0b8-00b1-4f89-abcf-2ff77634df50 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:40:16.115093 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-696ce0b8-00b1-4f89-abcf-2ff77634df50 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:40:16.118000 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0116945-7111-49be-aa2e-0c7056873e4a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:40:16.119939 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0116945-7111-49be-aa2e-0c7056873e4a service neutron] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:40:16.127027 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:40:16.140405 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 816/1620] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 1 16:40:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 16:40:16.142324 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0116945-7111-49be-aa2e-0c7056873e4a service neutron] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:40:16.152537 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 805/1621] 10.222.0.57 () {60 vars in 1143 bytes} [Fri Jul 1 16:40:16 2022] POST /identity/v3/users => generated 301 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 16:40:16.155660 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696ce0b8-00b1-4f89-abcf-2ff77634df50 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:40:16.158460 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0116945-7111-49be-aa2e-0c7056873e4a service neutron] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:40:16.166552 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f2470e-4bb0-4b97-8c44-e95a474adbc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:40:16.167836 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f2470e-4bb0-4b97-8c44-e95a474adbc7 admin admin] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:40:16.168260 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f2470e-4bb0-4b97-8c44-e95a474adbc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:40:16.168684 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f2470e-4bb0-4b97-8c44-e95a474adbc7 admin admin] PATH_INFO: `/v3/users` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:40:16.169706 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f2470e-4bb0-4b97-8c44-e95a474adbc7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:40:16.172350 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f2470e-4bb0-4b97-8c44-e95a474adbc7 admin admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:40:16.189442 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696ce0b8-00b1-4f89-abcf-2ff77634df50 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:40:16.205804 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 817/1622] 10.222.0.57 () {60 vars in 1143 bytes} [Fri Jul 1 16:40:16 2022] POST /identity/v3/users => generated 292 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 16:40:16.208681 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f2470e-4bb0-4b97-8c44-e95a474adbc7 admin admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:40:16.209968 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57120a23-bf85-487d-ad1d-39e4e014b4e9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a50f745b31941b7a7265c0fc852f48e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0788e6725a84e57a4174284c1845755', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:40:16.210940 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57120a23-bf85-487d-ad1d-39e4e014b4e9 service nova] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:40:16.211287 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57120a23-bf85-487d-ad1d-39e4e014b4e9 service nova] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:40:16.211589 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57120a23-bf85-487d-ad1d-39e4e014b4e9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:40:16.227952 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f2470e-4bb0-4b97-8c44-e95a474adbc7 admin admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:40:16.236454 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a14c0e-e109-4f18-bce2-bd76a8fffb75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:40:16.237491 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a14c0e-e109-4f18-bce2-bd76a8fffb75 admin admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:40:16.238423 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a14c0e-e109-4f18-bce2-bd76a8fffb75 admin admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:40:16.239306 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a14c0e-e109-4f18-bce2-bd76a8fffb75 admin admin] PATH_INFO: `/v3/roles` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:40:16.240734 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a14c0e-e109-4f18-bce2-bd76a8fffb75 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:40:16.242465 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a14c0e-e109-4f18-bce2-bd76a8fffb75 admin admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:40:16.264120 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 818/1623] 10.222.0.57 () {58 vars in 1121 bytes} [Fri Jul 1 16:40:16 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 16:40:16.266382 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a14c0e-e109-4f18-bce2-bd76a8fffb75 admin admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:40:16.285767 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a14c0e-e109-4f18-bce2-bd76a8fffb75 admin admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:40:16.298693 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98d260f-8e1a-4a4f-b2d5-342dbb9e8211 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:40:16.300039 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98d260f-8e1a-4a4f-b2d5-342dbb9e8211 admin admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:40:16.301675 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98d260f-8e1a-4a4f-b2d5-342dbb9e8211 admin admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:40:16.302006 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98d260f-8e1a-4a4f-b2d5-342dbb9e8211 admin admin] PATH_INFO: `/v3/roles` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:40:16.303050 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f98d260f-8e1a-4a4f-b2d5-342dbb9e8211 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:40:16.305486 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f98d260f-8e1a-4a4f-b2d5-342dbb9e8211 admin admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:40:16.326701 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 819/1624] 10.222.0.57 () {58 vars in 1121 bytes} [Fri Jul 1 16:40:16 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 16:40:16.328165 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98d260f-8e1a-4a4f-b2d5-342dbb9e8211 admin admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:40:16.346817 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98d260f-8e1a-4a4f-b2d5-342dbb9e8211 admin admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:40:16.357904 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8038ed4b-9a5e-4485-ba30-729261e3d92c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:40:16.359126 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8038ed4b-9a5e-4485-ba30-729261e3d92c admin admin] REQUEST_METHOD: `PUT` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:40:16.359847 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8038ed4b-9a5e-4485-ba30-729261e3d92c admin admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:40:16.360130 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8038ed4b-9a5e-4485-ba30-729261e3d92c admin admin] PATH_INFO: `/v3/projects/201fdf547f1a47f0b23307bd41a862a3/users/66e03519106646849671aa3699b3fd7b/roles/84ee739146934c1ab32c501ff8093644` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:40:16.381966 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57120a23-bf85-487d-ad1d-39e4e014b4e9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:40:16.385849 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57120a23-bf85-487d-ad1d-39e4e014b4e9 service nova] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:40:16.389346 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8038ed4b-9a5e-4485-ba30-729261e3d92c admin admin] RBAC: Authorizing `identity:create_grant(project_id=201fdf547f1a47f0b23307bd41a862a3, user_id=66e03519106646849671aa3699b3fd7b, role_id=84ee739146934c1ab32c501ff8093644)` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:40:16.396660 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8038ed4b-9a5e-4485-ba30-729261e3d92c admin admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:40:16.416831 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 806/1625] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 1 16:40:16 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 264 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 16:40:16.421003 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57120a23-bf85-487d-ad1d-39e4e014b4e9 service nova] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:40:16.436863 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 820/1626] 10.222.0.57 () {60 vars in 1482 bytes} [Fri Jul 1 16:40:16 2022] PUT /identity/v3/projects/201fdf547f1a47f0b23307bd41a862a3/users/66e03519106646849671aa3699b3fd7b/roles/84ee739146934c1ab32c501ff8093644 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 16:40:16.438506 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8038ed4b-9a5e-4485-ba30-729261e3d92c admin admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:40:16.442104 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57120a23-bf85-487d-ad1d-39e4e014b4e9 service nova] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:40:16.571010 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd54a265-5fb1-4b7d-9ccb-fe9b08ca95c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:40:16.572127 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd54a265-5fb1-4b7d-9ccb-fe9b08ca95c2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:40:16.572416 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd54a265-5fb1-4b7d-9ccb-fe9b08ca95c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:40:16.572702 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd54a265-5fb1-4b7d-9ccb-fe9b08ca95c2 admin admin] PATH_INFO: `/v3/projects/7dba4f4e07aa432d8abb893358db3f18/users/a497158ec38f4e7c8a1a53bac53273c3/roles/84ee739146934c1ab32c501ff8093644` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:40:16.593354 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8038ed4b-9a5e-4485-ba30-729261e3d92c admin admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:40:16.596510 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd54a265-5fb1-4b7d-9ccb-fe9b08ca95c2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7dba4f4e07aa432d8abb893358db3f18, user_id=a497158ec38f4e7c8a1a53bac53273c3, role_id=84ee739146934c1ab32c501ff8093644)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:40:16.599659 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd54a265-5fb1-4b7d-9ccb-fe9b08ca95c2 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:40:16.607200 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c37ba500-96df-41e0-9d5c-064822362867 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:40:16.610562 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37ba500-96df-41e0-9d5c-064822362867 admin admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:40:16.610923 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37ba500-96df-41e0-9d5c-064822362867 admin admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:40:16.611200 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37ba500-96df-41e0-9d5c-064822362867 admin admin] PATH_INFO: `/v3/roles` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:40:16.611981 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c37ba500-96df-41e0-9d5c-064822362867 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:40:16.619841 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c37ba500-96df-41e0-9d5c-064822362867 admin admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:40:16.625485 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 807/1627] 10.222.0.57 () {60 vars in 1482 bytes} [Fri Jul 1 16:40:16 2022] PUT /identity/v3/projects/7dba4f4e07aa432d8abb893358db3f18/users/a497158ec38f4e7c8a1a53bac53273c3/roles/84ee739146934c1ab32c501ff8093644 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 16:40:16.630632 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd54a265-5fb1-4b7d-9ccb-fe9b08ca95c2 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:40:16.646479 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 821/1628] 10.222.0.57 () {58 vars in 1121 bytes} [Fri Jul 1 16:40:16 2022] GET /identity/v3/roles => generated 1420 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 16:40:16.652020 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c37ba500-96df-41e0-9d5c-064822362867 admin admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:40:16.659090 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:40:16.684526 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c37ba500-96df-41e0-9d5c-064822362867 admin admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:40:16.754430 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd54a265-5fb1-4b7d-9ccb-fe9b08ca95c2 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:40:16.847617 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51513106-c9ee-4cec-9729-a8618c33f652 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:40:16.847617 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51513106-c9ee-4cec-9729-a8618c33f652 admin admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:40:16.847617 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51513106-c9ee-4cec-9729-a8618c33f652 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:40:16.847617 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51513106-c9ee-4cec-9729-a8618c33f652 admin admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:40:16.848782 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8ac91f-8e5e-43ab-88c5-5d660bb241b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:40:16.849372 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51513106-c9ee-4cec-9729-a8618c33f652 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:40:16.849467 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8ac91f-8e5e-43ab-88c5-5d660bb241b2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:40:16.849467 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8ac91f-8e5e-43ab-88c5-5d660bb241b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:40:16.849467 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8ac91f-8e5e-43ab-88c5-5d660bb241b2 admin admin] PATH_INFO: `/v3/projects/201fdf547f1a47f0b23307bd41a862a3/users/66e03519106646849671aa3699b3fd7b/roles/84ee739146934c1ab32c501ff8093644` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:40:16.850176 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51513106-c9ee-4cec-9729-a8618c33f652 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:40:16.850256 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8ac91f-8e5e-43ab-88c5-5d660bb241b2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=201fdf547f1a47f0b23307bd41a862a3, user_id=66e03519106646849671aa3699b3fd7b, role_id=84ee739146934c1ab32c501ff8093644)` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:40:16.850256 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8ac91f-8e5e-43ab-88c5-5d660bb241b2 admin admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:40:16.850256 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 822/1629] 10.222.0.57 () {60 vars in 1482 bytes} [Fri Jul 1 16:40:16 2022] PUT /identity/v3/projects/201fdf547f1a47f0b23307bd41a862a3/users/66e03519106646849671aa3699b3fd7b/roles/84ee739146934c1ab32c501ff8093644 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 16:40:16.850256 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54663408-e6b3-47b6-aafc-519e6b67ef70 None None] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:40:16.850256 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54663408-e6b3-47b6-aafc-519e6b67ef70 None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:40:16.850256 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54663408-e6b3-47b6-aafc-519e6b67ef70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:40:16.853476 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 808/1630] 10.222.0.57 () {58 vars in 1121 bytes} [Fri Jul 1 16:40:16 2022] GET /identity/v3/roles => generated 1420 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 16:40:16.853476 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51513106-c9ee-4cec-9729-a8618c33f652 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:40:16.858298 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51513106-c9ee-4cec-9729-a8618c33f652 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:40:16.873965 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.auth.core [None req-54663408-e6b3-47b6-aafc-519e6b67ef70 None None] MFA Rules not processed for user `66e03519106646849671aa3699b3fd7b`. Rule list: `[]` (Enabled: `True`). {{(pid=80312) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:40:16.875484 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-54663408-e6b3-47b6-aafc-519e6b67ef70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:40:16.907093 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:40:17.007391 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787d495a-47fe-45bb-be20-eae6486239d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:40:17.008302 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787d495a-47fe-45bb-be20-eae6486239d9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:40:17.008646 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787d495a-47fe-45bb-be20-eae6486239d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:40:17.008940 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787d495a-47fe-45bb-be20-eae6486239d9 admin admin] PATH_INFO: `/v3/projects/7dba4f4e07aa432d8abb893358db3f18/users/a497158ec38f4e7c8a1a53bac53273c3/roles/84ee739146934c1ab32c501ff8093644` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:40:17.024771 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 823/1631] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 1 16:40:16 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 16:40:17.026691 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104db5ff-8569-4271-ad26-ba2794abf935 None None] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:40:17.026994 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104db5ff-8569-4271-ad26-ba2794abf935 None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:40:17.027283 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104db5ff-8569-4271-ad26-ba2794abf935 None None] PATH_INFO: `/` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:40:17.029065 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 824/1632] 10.222.0.57 () {54 vars in 912 bytes} [Fri Jul 1 16:40:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 16:40:17.029782 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-787d495a-47fe-45bb-be20-eae6486239d9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7dba4f4e07aa432d8abb893358db3f18, user_id=a497158ec38f4e7c8a1a53bac53273c3, role_id=84ee739146934c1ab32c501ff8093644)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:40:17.029906 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104db5ff-8569-4271-ad26-ba2794abf935 None None] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:40:17.041733 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-787d495a-47fe-45bb-be20-eae6486239d9 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:40:17.088138 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 809/1633] 10.222.0.57 () {60 vars in 1482 bytes} [Fri Jul 1 16:40:16 2022] PUT /identity/v3/projects/7dba4f4e07aa432d8abb893358db3f18/users/a497158ec38f4e7c8a1a53bac53273c3/roles/84ee739146934c1ab32c501ff8093644 => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 16:40:17.093217 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1ccbc1-b7af-41e0-99c0-825775de246a None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:40:17.093511 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1ccbc1-b7af-41e0-99c0-825775de246a None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:40:17.093798 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1ccbc1-b7af-41e0-99c0-825775de246a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:40:17.130073 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104db5ff-8569-4271-ad26-ba2794abf935 None None] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:40:17.135774 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-ea1ccbc1-b7af-41e0-99c0-825775de246a None None] MFA Rules not processed for user `7a50f745b31941b7a7265c0fc852f48e`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:40:17.137017 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-ea1ccbc1-b7af-41e0-99c0-825775de246a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:40:17.250564 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dacbba6-63b7-4727-9f16-c3459c26aad2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbddf971a2354bbdbb9da1d63f3abf88', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0788e6725a84e57a4174284c1845755', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:40:17.251470 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dacbba6-63b7-4727-9f16-c3459c26aad2 service cinder] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:40:17.252083 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dacbba6-63b7-4727-9f16-c3459c26aad2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:40:17.252642 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dacbba6-63b7-4727-9f16-c3459c26aad2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:40:17.298019 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 810/1634] 10.222.0.57 () {58 vars in 1009 bytes} [Fri Jul 1 16:40:17 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 16:40:17.299370 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a784ca85-3e62-45a5-85f1-ed6274d8596e None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:40:17.299659 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a784ca85-3e62-45a5-85f1-ed6274d8596e None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:40:17.299935 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a784ca85-3e62-45a5-85f1-ed6274d8596e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:40:17.367540 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dacbba6-63b7-4727-9f16-c3459c26aad2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:40:17.369020 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dacbba6-63b7-4727-9f16-c3459c26aad2 service cinder] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:40:17.382607 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-a784ca85-3e62-45a5-85f1-ed6274d8596e None None] MFA Rules not processed for user `a497158ec38f4e7c8a1a53bac53273c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:40:17.383436 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 825/1635] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 1 16:40:17 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 16:40:17.384201 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dacbba6-63b7-4727-9f16-c3459c26aad2 service cinder] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:40:17.385981 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-a784ca85-3e62-45a5-85f1-ed6274d8596e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:40:17.399005 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dacbba6-63b7-4727-9f16-c3459c26aad2 service cinder] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:40:17.404579 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6912007-b6a2-4e87-a233-524085b02a7e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbddf971a2354bbdbb9da1d63f3abf88', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0788e6725a84e57a4174284c1845755', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:40:17.405610 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6912007-b6a2-4e87-a233-524085b02a7e service cinder] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:40:17.405914 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6912007-b6a2-4e87-a233-524085b02a7e service cinder] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:40:17.406242 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6912007-b6a2-4e87-a233-524085b02a7e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:40:17.444131 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6912007-b6a2-4e87-a233-524085b02a7e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:40:17.448917 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6912007-b6a2-4e87-a233-524085b02a7e service cinder] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:40:17.457126 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:40:17.494081 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 826/1636] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 1 16:40:17 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 16:40:17.594005 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 811/1637] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 1 16:40:17 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 295 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 16:40:17.615237 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6912007-b6a2-4e87-a233-524085b02a7e service cinder] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:40:17.623122 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=100 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:40:17.676936 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a784ca85-3e62-45a5-85f1-ed6274d8596e None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:40:17.690084 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a784ca85-3e62-45a5-85f1-ed6274d8596e None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:40:17.697272 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6abc49f8-d073-427f-94d6-e8c628a5c791 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbddf971a2354bbdbb9da1d63f3abf88', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0788e6725a84e57a4174284c1845755', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:40:17.698580 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abc49f8-d073-427f-94d6-e8c628a5c791 service cinder] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:40:17.699244 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abc49f8-d073-427f-94d6-e8c628a5c791 service cinder] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:40:17.699766 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abc49f8-d073-427f-94d6-e8c628a5c791 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:40:17.716299 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6abc49f8-d073-427f-94d6-e8c628a5c791 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:40:17.718361 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6abc49f8-d073-427f-94d6-e8c628a5c791 service cinder] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:40:17.755426 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 812/1638] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 1 16:40:17 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 79 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 01 16:40:17.799922 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6912007-b6a2-4e87-a233-524085b02a7e service cinder] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:40:17.809474 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698142d9-b878-49a5-86f5-9d300606fb31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:40:17.810803 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698142d9-b878-49a5-86f5-9d300606fb31 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:40:17.811401 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698142d9-b878-49a5-86f5-9d300606fb31 admin admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:40:17.811723 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698142d9-b878-49a5-86f5-9d300606fb31 admin admin] PATH_INFO: `/v3/users/dd8e8f465ad14b7c9e1340bfe18d4ff2` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:40:17.813797 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-698142d9-b878-49a5-86f5-9d300606fb31 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dd8e8f465ad14b7c9e1340bfe18d4ff2)` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:40:17.819189 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-698142d9-b878-49a5-86f5-9d300606fb31 admin admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:40:17.951973 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-698142d9-b878-49a5-86f5-9d300606fb31 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dd8e8f465ad14b7c9e1340bfe18d4ff2'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:40:17.984875 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-698142d9-b878-49a5-86f5-9d300606fb31 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dd8e8f465ad14b7c9e1340bfe18d4ff2'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:40:18.007476 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-698142d9-b878-49a5-86f5-9d300606fb31 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dd8e8f465ad14b7c9e1340bfe18d4ff2'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:40:18.011828 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-698142d9-b878-49a5-86f5-9d300606fb31 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dd8e8f465ad14b7c9e1340bfe18d4ff2'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:40:18.012907 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-698142d9-b878-49a5-86f5-9d300606fb31 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dd8e8f465ad14b7c9e1340bfe18d4ff2'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:40:18.065743 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 827/1639] 10.222.0.57 () {58 vars in 1223 bytes} [Fri Jul 1 16:40:17 2022] DELETE /identity/v3/users/dd8e8f465ad14b7c9e1340bfe18d4ff2 => generated 0 bytes in 450 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 16:40:18.068216 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46372ab1-eb61-4cf7-85c7-1f82eb2c4df3 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:40:18.068762 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46372ab1-eb61-4cf7-85c7-1f82eb2c4df3 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:40:18.068860 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46372ab1-eb61-4cf7-85c7-1f82eb2c4df3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:40:18.105889 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-46372ab1-eb61-4cf7-85c7-1f82eb2c4df3 None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:40:18.108725 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-46372ab1-eb61-4cf7-85c7-1f82eb2c4df3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:40:18.270100 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 813/1640] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 1 16:40:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 16:40:18.317555 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698142d9-b878-49a5-86f5-9d300606fb31 admin admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:40:18.319398 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-698142d9-b878-49a5-86f5-9d300606fb31 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:40:18.509547 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698142d9-b878-49a5-86f5-9d300606fb31 admin admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:40:18.517679 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d972647d-1951-488d-b230-c7e5728e67a8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ed200ce29684c5f80354fa9a8638fe2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0788e6725a84e57a4174284c1845755', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:40:18.518371 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d972647d-1951-488d-b230-c7e5728e67a8 service neutron] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:40:18.518662 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d972647d-1951-488d-b230-c7e5728e67a8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:40:18.519125 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d972647d-1951-488d-b230-c7e5728e67a8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:40:18.557671 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d972647d-1951-488d-b230-c7e5728e67a8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:40:18.559301 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d972647d-1951-488d-b230-c7e5728e67a8 service neutron] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:40:18.585722 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 828/1641] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 1 16:40:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 270 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 16:40:18.967614 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46372ab1-eb61-4cf7-85c7-1f82eb2c4df3 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:40:18.969039 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-46372ab1-eb61-4cf7-85c7-1f82eb2c4df3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:40:19.001421 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46372ab1-eb61-4cf7-85c7-1f82eb2c4df3 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:40:19.024187 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f201a1-0635-495a-bdba-b245532e0926 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:40:19.025637 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f201a1-0635-495a-bdba-b245532e0926 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:40:19.025637 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f201a1-0635-495a-bdba-b245532e0926 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:40:19.025866 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f201a1-0635-495a-bdba-b245532e0926 admin admin] PATH_INFO: `/v3/users/95ee9f7b04e240d28946d6cb8c24a8d8` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:40:19.035980 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f201a1-0635-495a-bdba-b245532e0926 admin admin] RBAC: Authorizing `identity:delete_user(user_id=95ee9f7b04e240d28946d6cb8c24a8d8)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:40:19.037036 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: WARNING py.warnings [None req-a3f201a1-0635-495a-bdba-b245532e0926 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 01 16:40:19.037036 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: warnings.warn(msg) Jul 01 16:40:19.037036 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: Jul 01 16:40:19.038417 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f201a1-0635-495a-bdba-b245532e0926 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:40:19.183576 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-a3f201a1-0635-495a-bdba-b245532e0926 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '95ee9f7b04e240d28946d6cb8c24a8d8'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:40:19.186885 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-a3f201a1-0635-495a-bdba-b245532e0926 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '95ee9f7b04e240d28946d6cb8c24a8d8'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:40:19.187817 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-a3f201a1-0635-495a-bdba-b245532e0926 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '95ee9f7b04e240d28946d6cb8c24a8d8'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:40:19.207830 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-a3f201a1-0635-495a-bdba-b245532e0926 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '95ee9f7b04e240d28946d6cb8c24a8d8'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:40:19.252168 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-a3f201a1-0635-495a-bdba-b245532e0926 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '95ee9f7b04e240d28946d6cb8c24a8d8'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:40:19.282465 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 814/1642] 10.222.0.57 () {58 vars in 1223 bytes} [Fri Jul 1 16:40:18 2022] DELETE /identity/v3/users/95ee9f7b04e240d28946d6cb8c24a8d8 => generated 0 bytes in 315 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 16:40:19.287960 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ab07d6-5d29-4597-b89b-65444dbc5d82 None None] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:40:19.288485 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ab07d6-5d29-4597-b89b-65444dbc5d82 None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:40:19.288485 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ab07d6-5d29-4597-b89b-65444dbc5d82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:40:19.322721 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.auth.core [None req-c5ab07d6-5d29-4597-b89b-65444dbc5d82 None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80312) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:40:19.325217 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-c5ab07d6-5d29-4597-b89b-65444dbc5d82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:40:19.375839 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:40:19.387498 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e365c46-e312-444c-b9dc-d3613d598b67 None None] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:40:19.387846 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e365c46-e312-444c-b9dc-d3613d598b67 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:40:19.388320 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e365c46-e312-444c-b9dc-d3613d598b67 None None] PATH_INFO: `/` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:40:19.389444 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 815/1643] 10.222.0.57 () {54 vars in 961 bytes} [Fri Jul 1 16:40:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 16:40:19.413348 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0987291f-3267-46da-8ed0-5c96efe9da6f None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:40:19.413687 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0987291f-3267-46da-8ed0-5c96efe9da6f None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:40:19.413908 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0987291f-3267-46da-8ed0-5c96efe9da6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:40:19.450424 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-0987291f-3267-46da-8ed0-5c96efe9da6f None None] MFA Rules not processed for user `7a50f745b31941b7a7265c0fc852f48e`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:40:19.456794 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-0987291f-3267-46da-8ed0-5c96efe9da6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:40:19.491147 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:40:19.670133 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 816/1644] 10.222.0.57 () {58 vars in 1058 bytes} [Fri Jul 1 16:40:19 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 256 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 01 16:40:19.679435 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d42ff1b-d115-41ee-b717-c96e901c7ea1 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:40:19.679885 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d42ff1b-d115-41ee-b717-c96e901c7ea1 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:40:19.680303 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d42ff1b-d115-41ee-b717-c96e901c7ea1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:40:19.744850 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-2d42ff1b-d115-41ee-b717-c96e901c7ea1 None None] MFA Rules not processed for user `f5ce7822f6b2469cba93544928f7957f`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:40:19.746109 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-2d42ff1b-d115-41ee-b717-c96e901c7ea1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:40:19.863555 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 829/1645] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 1 16:40:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 576 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 16:40:19.865318 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795f2fc3-0be2-4405-b8c6-dfb2e6ed70da None None] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:40:19.868341 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795f2fc3-0be2-4405-b8c6-dfb2e6ed70da None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:40:19.868984 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795f2fc3-0be2-4405-b8c6-dfb2e6ed70da None None] PATH_INFO: `/` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:40:19.872117 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 830/1646] 10.222.0.57 () {54 vars in 912 bytes} [Fri Jul 1 16:40:19 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 16:40:19.874234 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795f2fc3-0be2-4405-b8c6-dfb2e6ed70da None None] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:40:19.878867 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-795f2fc3-0be2-4405-b8c6-dfb2e6ed70da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:40:19.900656 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 817/1647] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 1 16:40:19 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 231 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 16:40:19.902904 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffd02e4-88ea-4514-befa-6ad3046af2ab None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:40:19.903205 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffd02e4-88ea-4514-befa-6ad3046af2ab None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:40:19.903543 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffd02e4-88ea-4514-befa-6ad3046af2ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:40:19.917004 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795f2fc3-0be2-4405-b8c6-dfb2e6ed70da None None] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:40:19.929296 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e49fa99-ccf5-4ce2-8fa2-279a19e43314 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:40:19.930470 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e49fa99-ccf5-4ce2-8fa2-279a19e43314 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:40:19.930909 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e49fa99-ccf5-4ce2-8fa2-279a19e43314 admin admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:40:19.931412 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e49fa99-ccf5-4ce2-8fa2-279a19e43314 admin admin] PATH_INFO: `/v3/projects/1fabfb2a97fa4894901bd3d193b3387d` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:40:19.933964 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e49fa99-ccf5-4ce2-8fa2-279a19e43314 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1fabfb2a97fa4894901bd3d193b3387d)` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:40:19.938597 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: WARNING py.warnings [None req-4e49fa99-ccf5-4ce2-8fa2-279a19e43314 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 01 16:40:19.938597 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: warnings.warn(msg) Jul 01 16:40:19.938597 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: Jul 01 16:40:19.940332 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e49fa99-ccf5-4ce2-8fa2-279a19e43314 admin admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:40:19.941363 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-5ffd02e4-88ea-4514-befa-6ad3046af2ab None None] MFA Rules not processed for user `7a50f745b31941b7a7265c0fc852f48e`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:40:19.945271 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-5ffd02e4-88ea-4514-befa-6ad3046af2ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:40:19.969832 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 818/1648] 10.222.0.57 () {58 vars in 1009 bytes} [Fri Jul 1 16:40:19 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 68 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 16:40:19.971774 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ffd02e4-88ea-4514-befa-6ad3046af2ab None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:40:19.977802 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-5ffd02e4-88ea-4514-befa-6ad3046af2ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:40:19.999681 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ffd02e4-88ea-4514-befa-6ad3046af2ab None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:40:20.010380 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4854c2c-7b9e-44c5-8bfe-b7b99df46af6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:40:20.011380 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4854c2c-7b9e-44c5-8bfe-b7b99df46af6 admin admin] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:40:20.011773 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4854c2c-7b9e-44c5-8bfe-b7b99df46af6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:40:20.012219 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4854c2c-7b9e-44c5-8bfe-b7b99df46af6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:40:20.013170 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4854c2c-7b9e-44c5-8bfe-b7b99df46af6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:40:20.017156 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4854c2c-7b9e-44c5-8bfe-b7b99df46af6 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:40:20.078215 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-4e49fa99-ccf5-4ce2-8fa2-279a19e43314 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1fabfb2a97fa4894901bd3d193b3387d', 'request_id': 'req-4e49fa99-ccf5-4ce2-8fa2-279a19e43314'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:40:20.083579 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 819/1649] 10.222.0.57 () {60 vars in 1152 bytes} [Fri Jul 1 16:40:19 2022] POST /identity/v3/projects => generated 405 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 16:40:20.085680 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4854c2c-7b9e-44c5-8bfe-b7b99df46af6 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:40:20.089917 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-b4854c2c-7b9e-44c5-8bfe-b7b99df46af6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:40:20.100775 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-4e49fa99-ccf5-4ce2-8fa2-279a19e43314 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1fabfb2a97fa4894901bd3d193b3387d', 'request_id': 'req-4e49fa99-ccf5-4ce2-8fa2-279a19e43314'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:40:20.109148 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-4e49fa99-ccf5-4ce2-8fa2-279a19e43314 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1fabfb2a97fa4894901bd3d193b3387d', 'request_id': 'req-4e49fa99-ccf5-4ce2-8fa2-279a19e43314'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:40:20.150886 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-4e49fa99-ccf5-4ce2-8fa2-279a19e43314 admin admin] The token cache is being invalidate because project 1fabfb2a97fa4894901bd3d193b3387d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80312) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 01 16:40:20.151167 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-4e49fa99-ccf5-4ce2-8fa2-279a19e43314 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:40:20.153116 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-4e49fa99-ccf5-4ce2-8fa2-279a19e43314 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:40:20.160525 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 831/1650] 10.222.0.57 () {58 vars in 1232 bytes} [Fri Jul 1 16:40:19 2022] DELETE /identity/v3/projects/1fabfb2a97fa4894901bd3d193b3387d => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 16:40:20.162266 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e49fa99-ccf5-4ce2-8fa2-279a19e43314 admin admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:40:20.165531 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-4e49fa99-ccf5-4ce2-8fa2-279a19e43314 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:40:20.248844 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4854c2c-7b9e-44c5-8bfe-b7b99df46af6 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:40:20.257913 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab30a023-29ea-4450-9552-7e81af1bb631 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbddf971a2354bbdbb9da1d63f3abf88', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0788e6725a84e57a4174284c1845755', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:40:20.258610 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab30a023-29ea-4450-9552-7e81af1bb631 service cinder] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:40:20.258896 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab30a023-29ea-4450-9552-7e81af1bb631 service cinder] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:40:20.259227 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab30a023-29ea-4450-9552-7e81af1bb631 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:40:20.263791 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-ab30a023-29ea-4450-9552-7e81af1bb631 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:40:20.315914 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e49fa99-ccf5-4ce2-8fa2-279a19e43314 admin admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:40:20.323863 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:40:20.433103 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab30a023-29ea-4450-9552-7e81af1bb631 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:40:20.435559 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab30a023-29ea-4450-9552-7e81af1bb631 service cinder] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:40:20.444874 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef2e139-8586-4e46-b2c4-b6af05b08322 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ed200ce29684c5f80354fa9a8638fe2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0788e6725a84e57a4174284c1845755', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:40:20.445703 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef2e139-8586-4e46-b2c4-b6af05b08322 service neutron] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:40:20.446092 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef2e139-8586-4e46-b2c4-b6af05b08322 service neutron] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:40:20.446802 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef2e139-8586-4e46-b2c4-b6af05b08322 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:40:20.451223 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-5ef2e139-8586-4e46-b2c4-b6af05b08322 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:40:20.468490 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 820/1651] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 1 16:40:20 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 384 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 16:40:20.495394 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab30a023-29ea-4450-9552-7e81af1bb631 service cinder] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:40:20.498956 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-ab30a023-29ea-4450-9552-7e81af1bb631 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:40:20.539695 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:40:20.713304 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef2e139-8586-4e46-b2c4-b6af05b08322 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:40:20.715095 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef2e139-8586-4e46-b2c4-b6af05b08322 service neutron] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:40:20.733552 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab30a023-29ea-4450-9552-7e81af1bb631 service cinder] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:40:20.742507 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 832/1652] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 1 16:40:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 581 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 16:40:20.743676 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef2e139-8586-4e46-b2c4-b6af05b08322 service neutron] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:40:20.746859 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-5ef2e139-8586-4e46-b2c4-b6af05b08322 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:40:20.750490 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1aaede-db76-4d2c-9498-496609593296 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:40:20.751191 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1aaede-db76-4d2c-9498-496609593296 admin admin] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:40:20.751481 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1aaede-db76-4d2c-9498-496609593296 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:40:20.751762 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1aaede-db76-4d2c-9498-496609593296 admin admin] PATH_INFO: `/v3/users` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:40:20.752521 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b1aaede-db76-4d2c-9498-496609593296 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:40:20.757390 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b1aaede-db76-4d2c-9498-496609593296 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:40:20.763678 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef2e139-8586-4e46-b2c4-b6af05b08322 service neutron] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:40:20.770696 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec01929-f06b-4d3e-a68f-a8f5a4add270 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:40:20.776027 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec01929-f06b-4d3e-a68f-a8f5a4add270 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:40:20.777534 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec01929-f06b-4d3e-a68f-a8f5a4add270 admin admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:40:20.777902 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec01929-f06b-4d3e-a68f-a8f5a4add270 admin admin] PATH_INFO: `/v3/users/66e03519106646849671aa3699b3fd7b` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:40:20.779600 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec01929-f06b-4d3e-a68f-a8f5a4add270 admin admin] RBAC: Authorizing `identity:delete_user(user_id=66e03519106646849671aa3699b3fd7b)` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:40:20.782640 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec01929-f06b-4d3e-a68f-a8f5a4add270 admin admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:40:20.806817 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 821/1653] 10.222.0.57 () {60 vars in 1143 bytes} [Fri Jul 1 16:40:20 2022] POST /identity/v3/users => generated 309 bytes in 338 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 16:40:20.814317 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1aaede-db76-4d2c-9498-496609593296 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:40:20.849164 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1aaede-db76-4d2c-9498-496609593296 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:40:20.864145 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f7cda2-5008-4637-a441-317e55c3c075 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:40:20.867142 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f7cda2-5008-4637-a441-317e55c3c075 admin admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:40:20.867507 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f7cda2-5008-4637-a441-317e55c3c075 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:40:20.869031 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f7cda2-5008-4637-a441-317e55c3c075 admin admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:40:20.869839 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f7cda2-5008-4637-a441-317e55c3c075 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:40:20.877861 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f7cda2-5008-4637-a441-317e55c3c075 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:40:20.910663 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 822/1654] 10.222.0.57 () {58 vars in 1121 bytes} [Fri Jul 1 16:40:20 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 16:40:20.911744 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f7cda2-5008-4637-a441-317e55c3c075 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:40:20.920702 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-aec01929-f06b-4d3e-a68f-a8f5a4add270 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '66e03519106646849671aa3699b3fd7b'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:40:20.952142 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f7cda2-5008-4637-a441-317e55c3c075 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:40:20.966021 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5af0d9-d643-472e-99cf-325ef9910085 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:40:20.970064 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5af0d9-d643-472e-99cf-325ef9910085 admin admin] REQUEST_METHOD: `PUT` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:40:20.970064 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5af0d9-d643-472e-99cf-325ef9910085 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:40:20.971474 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5af0d9-d643-472e-99cf-325ef9910085 admin admin] PATH_INFO: `/v3/projects/53e556371eec40d398ae089d2714a856/users/e5e4c87a0666464bbf3675103195e41d/roles/84ee739146934c1ab32c501ff8093644` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:40:20.971592 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-aec01929-f06b-4d3e-a68f-a8f5a4add270 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '66e03519106646849671aa3699b3fd7b'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:40:21.001060 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-aec01929-f06b-4d3e-a68f-a8f5a4add270 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '66e03519106646849671aa3699b3fd7b'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:40:21.003758 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-aec01929-f06b-4d3e-a68f-a8f5a4add270 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '66e03519106646849671aa3699b3fd7b'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:40:21.008112 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-aec01929-f06b-4d3e-a68f-a8f5a4add270 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '66e03519106646849671aa3699b3fd7b'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:40:21.026976 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e5af0d9-d643-472e-99cf-325ef9910085 admin admin] RBAC: Authorizing `identity:create_grant(project_id=53e556371eec40d398ae089d2714a856, user_id=e5e4c87a0666464bbf3675103195e41d, role_id=84ee739146934c1ab32c501ff8093644)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:40:21.043758 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e5af0d9-d643-472e-99cf-325ef9910085 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:40:21.051617 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 833/1655] 10.222.0.57 () {58 vars in 1223 bytes} [Fri Jul 1 16:40:20 2022] DELETE /identity/v3/users/66e03519106646849671aa3699b3fd7b => generated 0 bytes in 308 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 16:40:21.090244 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 823/1656] 10.222.0.57 () {60 vars in 1482 bytes} [Fri Jul 1 16:40:20 2022] PUT /identity/v3/projects/53e556371eec40d398ae089d2714a856/users/e5e4c87a0666464bbf3675103195e41d/roles/84ee739146934c1ab32c501ff8093644 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 16:40:21.091159 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec01929-f06b-4d3e-a68f-a8f5a4add270 admin admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:40:21.092834 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-aec01929-f06b-4d3e-a68f-a8f5a4add270 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:40:21.320783 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec01929-f06b-4d3e-a68f-a8f5a4add270 admin admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:40:21.412219 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:40:21.480202 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0035c66c-b53d-4bfc-b3b8-247ba0aef67b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:40:21.482220 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0035c66c-b53d-4bfc-b3b8-247ba0aef67b admin admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:40:21.482836 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0035c66c-b53d-4bfc-b3b8-247ba0aef67b admin admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:40:21.484007 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0035c66c-b53d-4bfc-b3b8-247ba0aef67b admin admin] PATH_INFO: `/v3/roles` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:40:21.484949 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0035c66c-b53d-4bfc-b3b8-247ba0aef67b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:40:21.487109 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0035c66c-b53d-4bfc-b3b8-247ba0aef67b admin admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:40:21.516906 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 834/1657] 10.222.0.57 () {58 vars in 1121 bytes} [Fri Jul 1 16:40:21 2022] GET /identity/v3/roles => generated 1420 bytes in 426 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 01 16:40:21.517733 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5af0d9-d643-472e-99cf-325ef9910085 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:40:21.555587 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5af0d9-d643-472e-99cf-325ef9910085 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:40:21.571858 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172a2345-10bd-48a1-9e00-52ca89192c98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:40:21.573010 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172a2345-10bd-48a1-9e00-52ca89192c98 admin admin] REQUEST_METHOD: `PUT` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:40:21.573363 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172a2345-10bd-48a1-9e00-52ca89192c98 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:40:21.573840 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172a2345-10bd-48a1-9e00-52ca89192c98 admin admin] PATH_INFO: `/v3/projects/53e556371eec40d398ae089d2714a856/users/e5e4c87a0666464bbf3675103195e41d/roles/84ee739146934c1ab32c501ff8093644` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:40:21.577786 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-172a2345-10bd-48a1-9e00-52ca89192c98 admin admin] RBAC: Authorizing `identity:create_grant(project_id=53e556371eec40d398ae089d2714a856, user_id=e5e4c87a0666464bbf3675103195e41d, role_id=84ee739146934c1ab32c501ff8093644)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:40:21.585547 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-172a2345-10bd-48a1-9e00-52ca89192c98 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:40:21.626124 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 824/1658] 10.222.0.57 () {60 vars in 1482 bytes} [Fri Jul 1 16:40:21 2022] PUT /identity/v3/projects/53e556371eec40d398ae089d2714a856/users/e5e4c87a0666464bbf3675103195e41d/roles/84ee739146934c1ab32c501ff8093644 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 01 16:40:22.398597 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056b18c0-c22d-46a2-b8f5-8499f07588bc None None] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:40:22.399130 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056b18c0-c22d-46a2-b8f5-8499f07588bc None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:40:22.399209 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056b18c0-c22d-46a2-b8f5-8499f07588bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:40:22.444511 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.auth.core [None req-056b18c0-c22d-46a2-b8f5-8499f07588bc None None] MFA Rules not processed for user `7c4da21fbfa3472cac6e766a2f4799d8`. Rule list: `[]` (Enabled: `True`). {{(pid=80312) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:40:22.446187 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-056b18c0-c22d-46a2-b8f5-8499f07588bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:40:22.566640 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 835/1659] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 1 16:40:22 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 16:40:22.582724 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172a2345-10bd-48a1-9e00-52ca89192c98 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:40:22.585743 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-172a2345-10bd-48a1-9e00-52ca89192c98 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:40:22.663412 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:40:22.694789 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172a2345-10bd-48a1-9e00-52ca89192c98 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:40:22.720874 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0108fc14-ef6e-4a77-87b7-fa4e3764db0e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a50f745b31941b7a7265c0fc852f48e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0788e6725a84e57a4174284c1845755', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:40:22.721820 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0108fc14-ef6e-4a77-87b7-fa4e3764db0e service nova] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:40:22.722816 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0108fc14-ef6e-4a77-87b7-fa4e3764db0e service nova] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:40:22.723538 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0108fc14-ef6e-4a77-87b7-fa4e3764db0e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:40:22.735160 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0108fc14-ef6e-4a77-87b7-fa4e3764db0e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:40:22.738842 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0108fc14-ef6e-4a77-87b7-fa4e3764db0e service nova] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:40:22.772768 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 825/1660] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 1 16:40:22 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 191 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 16:40:22.888481 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056b18c0-c22d-46a2-b8f5-8499f07588bc None None] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:40:22.890071 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-056b18c0-c22d-46a2-b8f5-8499f07588bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:40:23.064567 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056b18c0-c22d-46a2-b8f5-8499f07588bc None None] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:40:23.073723 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93944d23-8547-4bc9-bc50-119d6f7b4827 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:40:23.074710 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93944d23-8547-4bc9-bc50-119d6f7b4827 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:40:23.075006 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93944d23-8547-4bc9-bc50-119d6f7b4827 admin admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:40:23.075291 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93944d23-8547-4bc9-bc50-119d6f7b4827 admin admin] PATH_INFO: `/v3/projects/fe48aa4179354b4fac35b7415ea47f28` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:40:23.076740 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93944d23-8547-4bc9-bc50-119d6f7b4827 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fe48aa4179354b4fac35b7415ea47f28)` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:40:23.081921 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93944d23-8547-4bc9-bc50-119d6f7b4827 admin admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:40:23.113145 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0108fc14-ef6e-4a77-87b7-fa4e3764db0e service nova] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:40:23.114812 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-0108fc14-ef6e-4a77-87b7-fa4e3764db0e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:40:23.142156 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0108fc14-ef6e-4a77-87b7-fa4e3764db0e service nova] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:40:23.169443 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-93944d23-8547-4bc9-bc50-119d6f7b4827 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fe48aa4179354b4fac35b7415ea47f28', 'request_id': 'req-93944d23-8547-4bc9-bc50-119d6f7b4827'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:40:23.186814 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-93944d23-8547-4bc9-bc50-119d6f7b4827 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fe48aa4179354b4fac35b7415ea47f28', 'request_id': 'req-93944d23-8547-4bc9-bc50-119d6f7b4827'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:40:23.198580 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:40:23.201123 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-93944d23-8547-4bc9-bc50-119d6f7b4827 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fe48aa4179354b4fac35b7415ea47f28', 'request_id': 'req-93944d23-8547-4bc9-bc50-119d6f7b4827'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:40:23.231323 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-93944d23-8547-4bc9-bc50-119d6f7b4827 admin admin] The token cache is being invalidate because project fe48aa4179354b4fac35b7415ea47f28 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80312) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 01 16:40:23.231629 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-93944d23-8547-4bc9-bc50-119d6f7b4827 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:40:23.232371 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-93944d23-8547-4bc9-bc50-119d6f7b4827 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:40:23.233992 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 836/1661] 10.222.0.57 () {58 vars in 1232 bytes} [Fri Jul 1 16:40:22 2022] DELETE /identity/v3/projects/fe48aa4179354b4fac35b7415ea47f28 => generated 0 bytes in 346 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 16:40:23.238442 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93944d23-8547-4bc9-bc50-119d6f7b4827 admin admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:40:23.242523 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-93944d23-8547-4bc9-bc50-119d6f7b4827 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:40:23.278871 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785a9769-1de2-4875-a2e5-dd4832b56f61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:40:23.281020 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785a9769-1de2-4875-a2e5-dd4832b56f61 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:40:23.281545 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785a9769-1de2-4875-a2e5-dd4832b56f61 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:40:23.283600 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785a9769-1de2-4875-a2e5-dd4832b56f61 admin admin] PATH_INFO: `/v3/projects/201fdf547f1a47f0b23307bd41a862a3` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:40:23.287862 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785a9769-1de2-4875-a2e5-dd4832b56f61 admin admin] RBAC: Authorizing `identity:delete_project(project_id=201fdf547f1a47f0b23307bd41a862a3)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:40:23.292873 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785a9769-1de2-4875-a2e5-dd4832b56f61 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:40:23.385098 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-785a9769-1de2-4875-a2e5-dd4832b56f61 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '201fdf547f1a47f0b23307bd41a862a3', 'request_id': 'req-785a9769-1de2-4875-a2e5-dd4832b56f61'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:40:23.389287 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93944d23-8547-4bc9-bc50-119d6f7b4827 admin admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:40:23.400773 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-785a9769-1de2-4875-a2e5-dd4832b56f61 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '201fdf547f1a47f0b23307bd41a862a3', 'request_id': 'req-785a9769-1de2-4875-a2e5-dd4832b56f61'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:40:23.429071 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-785a9769-1de2-4875-a2e5-dd4832b56f61 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '201fdf547f1a47f0b23307bd41a862a3', 'request_id': 'req-785a9769-1de2-4875-a2e5-dd4832b56f61'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:40:23.465119 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-785a9769-1de2-4875-a2e5-dd4832b56f61 admin admin] The token cache is being invalidate because project 201fdf547f1a47f0b23307bd41a862a3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80311) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 01 16:40:23.465411 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-785a9769-1de2-4875-a2e5-dd4832b56f61 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:40:23.466278 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-785a9769-1de2-4875-a2e5-dd4832b56f61 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:40:23.473887 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 826/1662] 10.222.0.57 () {58 vars in 1232 bytes} [Fri Jul 1 16:40:23 2022] DELETE /identity/v3/projects/201fdf547f1a47f0b23307bd41a862a3 => generated 0 bytes in 358 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 16:40:23.476739 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd47b5d-3608-48a2-a7b7-c8926eab71a0 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:40:23.477097 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd47b5d-3608-48a2-a7b7-c8926eab71a0 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:40:23.477986 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd47b5d-3608-48a2-a7b7-c8926eab71a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:40:23.485406 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d57d41-f5c3-4611-b3f5-4c4aa4c7a4db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:40:23.486509 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d57d41-f5c3-4611-b3f5-4c4aa4c7a4db admin admin] REQUEST_METHOD: `DELETE` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:40:23.486821 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d57d41-f5c3-4611-b3f5-4c4aa4c7a4db admin admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:40:23.487122 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d57d41-f5c3-4611-b3f5-4c4aa4c7a4db admin admin] PATH_INFO: `/v3/users/a497158ec38f4e7c8a1a53bac53273c3` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:40:23.488999 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d57d41-f5c3-4611-b3f5-4c4aa4c7a4db admin admin] RBAC: Authorizing `identity:delete_user(user_id=a497158ec38f4e7c8a1a53bac53273c3)` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:40:23.491338 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d57d41-f5c3-4611-b3f5-4c4aa4c7a4db admin admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:40:23.523949 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-ddd47b5d-3608-48a2-a7b7-c8926eab71a0 None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:40:23.528399 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-ddd47b5d-3608-48a2-a7b7-c8926eab71a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:40:23.546306 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 827/1663] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 1 16:40:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 72 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 16:40:23.555386 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1daaacfb-9968-4afc-8bb9-ea06e87c8acf None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:40:23.555557 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1daaacfb-9968-4afc-8bb9-ea06e87c8acf None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:40:23.555881 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1daaacfb-9968-4afc-8bb9-ea06e87c8acf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:40:23.562810 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-55d57d41-f5c3-4611-b3f5-4c4aa4c7a4db admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a497158ec38f4e7c8a1a53bac53273c3'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:40:23.582532 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-55d57d41-f5c3-4611-b3f5-4c4aa4c7a4db admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a497158ec38f4e7c8a1a53bac53273c3'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:40:23.591857 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-55d57d41-f5c3-4611-b3f5-4c4aa4c7a4db admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a497158ec38f4e7c8a1a53bac53273c3'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:40:23.592653 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-55d57d41-f5c3-4611-b3f5-4c4aa4c7a4db admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a497158ec38f4e7c8a1a53bac53273c3'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:40:23.593334 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-55d57d41-f5c3-4611-b3f5-4c4aa4c7a4db admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a497158ec38f4e7c8a1a53bac53273c3'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:40:23.595111 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-1daaacfb-9968-4afc-8bb9-ea06e87c8acf None None] MFA Rules not processed for user `e5e4c87a0666464bbf3675103195e41d`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:40:23.596728 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-1daaacfb-9968-4afc-8bb9-ea06e87c8acf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:40:23.618527 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 837/1664] 10.222.0.57 () {58 vars in 1223 bytes} [Fri Jul 1 16:40:23 2022] DELETE /identity/v3/users/a497158ec38f4e7c8a1a53bac53273c3 => generated 0 bytes in 378 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 01 16:40:23.618527 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791e7fc8-1fe9-438b-b090-7afe8695a6a9 None None] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:40:23.618527 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791e7fc8-1fe9-438b-b090-7afe8695a6a9 None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:40:23.618854 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791e7fc8-1fe9-438b-b090-7afe8695a6a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:40:23.646274 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.auth.core [None req-791e7fc8-1fe9-438b-b090-7afe8695a6a9 None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80312) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:40:23.647781 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-791e7fc8-1fe9-438b-b090-7afe8695a6a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:40:23.666362 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:40:23.680952 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:40:23.736359 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 828/1665] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 1 16:40:23 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 16:40:23.780453 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 838/1666] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 1 16:40:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 16:40:23.788583 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1daaacfb-9968-4afc-8bb9-ea06e87c8acf None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:40:23.806712 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1daaacfb-9968-4afc-8bb9-ea06e87c8acf None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:40:23.813043 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ca338a-2fa1-4e7b-bff9-c2f13f7653e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:40:23.813895 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ca338a-2fa1-4e7b-bff9-c2f13f7653e1 admin admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:40:23.814259 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ca338a-2fa1-4e7b-bff9-c2f13f7653e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:40:23.814639 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ca338a-2fa1-4e7b-bff9-c2f13f7653e1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:40:23.816350 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ca338a-2fa1-4e7b-bff9-c2f13f7653e1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:40:23.816757 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ca338a-2fa1-4e7b-bff9-c2f13f7653e1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:40:23.818904 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ca338a-2fa1-4e7b-bff9-c2f13f7653e1 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:40:23.854110 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 829/1667] 10.222.0.57 () {58 vars in 1152 bytes} [Fri Jul 1 16:40:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:40:23.858314 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791e7fc8-1fe9-438b-b090-7afe8695a6a9 None None] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:40:23.879350 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791e7fc8-1fe9-438b-b090-7afe8695a6a9 None None] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:40:23.885194 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd9c317-9774-4127-b72b-d92a304c6abd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:40:23.886842 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd9c317-9774-4127-b72b-d92a304c6abd admin admin] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:40:23.889854 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd9c317-9774-4127-b72b-d92a304c6abd admin admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:40:23.890796 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd9c317-9774-4127-b72b-d92a304c6abd admin admin] PATH_INFO: `/v3/projects` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:40:23.891732 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd9c317-9774-4127-b72b-d92a304c6abd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:40:23.896597 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd9c317-9774-4127-b72b-d92a304c6abd admin admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:40:23.953241 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 839/1668] 10.222.0.57 () {60 vars in 1152 bytes} [Fri Jul 1 16:40:23 2022] POST /identity/v3/projects => generated 409 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 01 16:40:23.958272 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ca338a-2fa1-4e7b-bff9-c2f13f7653e1 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:40:23.976006 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ca338a-2fa1-4e7b-bff9-c2f13f7653e1 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:40:24.090826 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc156f3-2151-4261-be41-db12cb3c8e56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:40:24.094531 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc156f3-2151-4261-be41-db12cb3c8e56 admin admin] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:40:24.095189 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc156f3-2151-4261-be41-db12cb3c8e56 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:40:24.095720 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc156f3-2151-4261-be41-db12cb3c8e56 admin admin] PATH_INFO: `/v3/users` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:40:24.096671 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc156f3-2151-4261-be41-db12cb3c8e56 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:40:24.100406 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc156f3-2151-4261-be41-db12cb3c8e56 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:40:24.149692 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 830/1669] 10.222.0.57 () {60 vars in 1143 bytes} [Fri Jul 1 16:40:23 2022] POST /identity/v3/users => generated 305 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 01 16:40:24.156841 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd9c317-9774-4127-b72b-d92a304c6abd admin admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:40:24.178188 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd9c317-9774-4127-b72b-d92a304c6abd admin admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:40:24.184043 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5bab8f3-f51e-4090-882b-d2d7c64bcba5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:40:24.184869 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bab8f3-f51e-4090-882b-d2d7c64bcba5 admin admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:40:24.185262 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bab8f3-f51e-4090-882b-d2d7c64bcba5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:40:24.185648 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bab8f3-f51e-4090-882b-d2d7c64bcba5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:40:24.186990 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5bab8f3-f51e-4090-882b-d2d7c64bcba5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:40:24.189117 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5bab8f3-f51e-4090-882b-d2d7c64bcba5 admin admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:40:24.209297 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 840/1670] 10.222.0.57 () {58 vars in 1121 bytes} [Fri Jul 1 16:40:24 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 16:40:24.215950 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc156f3-2151-4261-be41-db12cb3c8e56 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:40:24.256457 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc156f3-2151-4261-be41-db12cb3c8e56 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:40:24.266205 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de669e4f-9f6d-4602-89f5-e3a63f77e216 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:40:24.266934 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de669e4f-9f6d-4602-89f5-e3a63f77e216 admin admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:40:24.267300 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de669e4f-9f6d-4602-89f5-e3a63f77e216 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:40:24.267516 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de669e4f-9f6d-4602-89f5-e3a63f77e216 admin admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:40:24.268274 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de669e4f-9f6d-4602-89f5-e3a63f77e216 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:40:24.271195 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de669e4f-9f6d-4602-89f5-e3a63f77e216 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:40:24.286435 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 831/1671] 10.222.0.57 () {58 vars in 1121 bytes} [Fri Jul 1 16:40:24 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 16:40:24.290011 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5bab8f3-f51e-4090-882b-d2d7c64bcba5 admin admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:40:24.307587 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5bab8f3-f51e-4090-882b-d2d7c64bcba5 admin admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:40:24.314865 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d5d368-6a3e-4ce1-a218-d3435efd6397 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:40:24.315963 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d5d368-6a3e-4ce1-a218-d3435efd6397 admin admin] REQUEST_METHOD: `PUT` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:40:24.316249 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d5d368-6a3e-4ce1-a218-d3435efd6397 admin admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:40:24.316529 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d5d368-6a3e-4ce1-a218-d3435efd6397 admin admin] PATH_INFO: `/v3/projects/d07b741a7f8e42c8b7d0c72ef29b6be1/users/923dd887fef04df1baa5377db1f6c5e1/roles/983be40bdab04c478eaa90c6b6a46b14` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:40:24.346128 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d5d368-6a3e-4ce1-a218-d3435efd6397 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d07b741a7f8e42c8b7d0c72ef29b6be1, user_id=923dd887fef04df1baa5377db1f6c5e1, role_id=983be40bdab04c478eaa90c6b6a46b14)` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:40:24.349609 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d5d368-6a3e-4ce1-a218-d3435efd6397 admin admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:40:24.390901 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 841/1672] 10.222.0.57 () {60 vars in 1482 bytes} [Fri Jul 1 16:40:24 2022] PUT /identity/v3/projects/d07b741a7f8e42c8b7d0c72ef29b6be1/users/923dd887fef04df1baa5377db1f6c5e1/roles/983be40bdab04c478eaa90c6b6a46b14 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 16:40:24.406285 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4acbf45-aca6-483b-aab6-ad6ab31c3465 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:40:24.406640 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4acbf45-aca6-483b-aab6-ad6ab31c3465 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:40:24.406955 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4acbf45-aca6-483b-aab6-ad6ab31c3465 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:40:24.443294 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-f4acbf45-aca6-483b-aab6-ad6ab31c3465 None None] MFA Rules not processed for user `923dd887fef04df1baa5377db1f6c5e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:40:24.444627 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-f4acbf45-aca6-483b-aab6-ad6ab31c3465 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:40:24.541984 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 832/1673] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 1 16:40:24 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 16:40:24.546622 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d5d368-6a3e-4ce1-a218-d3435efd6397 admin admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:40:24.639836 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d5d368-6a3e-4ce1-a218-d3435efd6397 admin admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:40:24.646182 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0018ffec-87a5-434d-9a6b-42074947bd05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:40:24.647140 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0018ffec-87a5-434d-9a6b-42074947bd05 admin admin] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:40:24.647526 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0018ffec-87a5-434d-9a6b-42074947bd05 admin admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:40:24.647927 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0018ffec-87a5-434d-9a6b-42074947bd05 admin admin] PATH_INFO: `/v3/projects` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:40:24.648776 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0018ffec-87a5-434d-9a6b-42074947bd05 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:40:24.651699 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0018ffec-87a5-434d-9a6b-42074947bd05 admin admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:40:24.691044 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4acbf45-aca6-483b-aab6-ad6ab31c3465 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:40:24.693557 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 842/1674] 10.222.0.57 () {60 vars in 1152 bytes} [Fri Jul 1 16:40:24 2022] POST /identity/v3/projects => generated 409 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 16:40:24.703763 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4acbf45-aca6-483b-aab6-ad6ab31c3465 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:40:24.759305 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:40:24.789396 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50537a05-7aa0-4b0e-9c79-b35ee63b42e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:40:24.790205 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50537a05-7aa0-4b0e-9c79-b35ee63b42e7 admin admin] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:40:24.790718 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50537a05-7aa0-4b0e-9c79-b35ee63b42e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:40:24.791301 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50537a05-7aa0-4b0e-9c79-b35ee63b42e7 admin admin] PATH_INFO: `/v3/users` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:40:24.792315 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50537a05-7aa0-4b0e-9c79-b35ee63b42e7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:40:24.795201 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50537a05-7aa0-4b0e-9c79-b35ee63b42e7 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:40:24.839128 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0018ffec-87a5-434d-9a6b-42074947bd05 admin admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:40:24.841985 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 833/1675] 10.222.0.57 () {60 vars in 1143 bytes} [Fri Jul 1 16:40:24 2022] POST /identity/v3/users => generated 311 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 01 16:40:24.853393 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0018ffec-87a5-434d-9a6b-42074947bd05 admin admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:40:24.861215 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7627f9a9-56ff-42b1-b101-a12c284c6fff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:40:24.862249 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7627f9a9-56ff-42b1-b101-a12c284c6fff admin admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:40:24.862992 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7627f9a9-56ff-42b1-b101-a12c284c6fff admin admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:40:24.864095 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7627f9a9-56ff-42b1-b101-a12c284c6fff admin admin] PATH_INFO: `/v3/roles` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:40:24.865418 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7627f9a9-56ff-42b1-b101-a12c284c6fff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:40:24.867354 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7627f9a9-56ff-42b1-b101-a12c284c6fff admin admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:40:24.883272 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 843/1676] 10.222.0.57 () {58 vars in 1121 bytes} [Fri Jul 1 16:40:24 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 16:40:24.887428 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50537a05-7aa0-4b0e-9c79-b35ee63b42e7 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:40:24.900254 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50537a05-7aa0-4b0e-9c79-b35ee63b42e7 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:40:24.907425 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d89718-f50b-446a-90aa-f52131f1006e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:40:24.908905 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d89718-f50b-446a-90aa-f52131f1006e admin admin] REQUEST_METHOD: `PUT` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:40:24.910221 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d89718-f50b-446a-90aa-f52131f1006e admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:40:24.912338 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d89718-f50b-446a-90aa-f52131f1006e admin admin] PATH_INFO: `/v3/projects/53eba863d47f4e669ef367658a83a96b/users/15e12b51d19c451d91be7bcecc6b66a1/roles/84ee739146934c1ab32c501ff8093644` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:40:24.914946 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7627f9a9-56ff-42b1-b101-a12c284c6fff admin admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:40:24.917067 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-7627f9a9-56ff-42b1-b101-a12c284c6fff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:40:24.948738 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7627f9a9-56ff-42b1-b101-a12c284c6fff admin admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:40:24.957273 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b77362-8feb-46e8-bffb-61136e04d851 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:40:24.961037 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d89718-f50b-446a-90aa-f52131f1006e admin admin] RBAC: Authorizing `identity:create_grant(project_id=53eba863d47f4e669ef367658a83a96b, user_id=15e12b51d19c451d91be7bcecc6b66a1, role_id=84ee739146934c1ab32c501ff8093644)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:40:24.963738 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d89718-f50b-446a-90aa-f52131f1006e admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:40:24.974328 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b77362-8feb-46e8-bffb-61136e04d851 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:40:24.978594 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b77362-8feb-46e8-bffb-61136e04d851 admin admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:40:24.979592 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b77362-8feb-46e8-bffb-61136e04d851 admin admin] PATH_INFO: `/v3/projects/7dba4f4e07aa432d8abb893358db3f18` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:40:24.983299 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b77362-8feb-46e8-bffb-61136e04d851 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7dba4f4e07aa432d8abb893358db3f18)` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:40:24.986422 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 834/1677] 10.222.0.57 () {60 vars in 1482 bytes} [Fri Jul 1 16:40:24 2022] PUT /identity/v3/projects/53eba863d47f4e669ef367658a83a96b/users/15e12b51d19c451d91be7bcecc6b66a1/roles/84ee739146934c1ab32c501ff8093644 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 16:40:24.991239 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b77362-8feb-46e8-bffb-61136e04d851 admin admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:40:24.994299 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d89718-f50b-446a-90aa-f52131f1006e admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:40:25.077362 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=95,keystone:INSERT=3,keystone:DELETE=2 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:40:25.117375 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-47b77362-8feb-46e8-bffb-61136e04d851 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7dba4f4e07aa432d8abb893358db3f18', 'request_id': 'req-47b77362-8feb-46e8-bffb-61136e04d851'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:40:25.127883 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-47b77362-8feb-46e8-bffb-61136e04d851 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7dba4f4e07aa432d8abb893358db3f18', 'request_id': 'req-47b77362-8feb-46e8-bffb-61136e04d851'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:40:25.149411 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d89718-f50b-446a-90aa-f52131f1006e admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:40:25.157978 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-47b77362-8feb-46e8-bffb-61136e04d851 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7dba4f4e07aa432d8abb893358db3f18', 'request_id': 'req-47b77362-8feb-46e8-bffb-61136e04d851'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:40:25.186498 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-47b77362-8feb-46e8-bffb-61136e04d851 admin admin] The token cache is being invalidate because project 7dba4f4e07aa432d8abb893358db3f18 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80312) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 01 16:40:25.187040 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-47b77362-8feb-46e8-bffb-61136e04d851 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:40:25.188255 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-47b77362-8feb-46e8-bffb-61136e04d851 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:40:25.190329 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 844/1678] 10.222.0.57 () {58 vars in 1232 bytes} [Fri Jul 1 16:40:24 2022] DELETE /identity/v3/projects/7dba4f4e07aa432d8abb893358db3f18 => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 01 16:40:25.197170 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180a2157-1a87-4c64-9c08-d6842db6a9ab None None] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:40:25.197682 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180a2157-1a87-4c64-9c08-d6842db6a9ab None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:40:25.199446 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180a2157-1a87-4c64-9c08-d6842db6a9ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:40:25.229103 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.auth.core [None req-180a2157-1a87-4c64-9c08-d6842db6a9ab None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80312) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:40:25.230614 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-180a2157-1a87-4c64-9c08-d6842db6a9ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:40:25.323199 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3ddfca-2867-4e1a-8023-1f0b7c15c4bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:40:25.325800 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3ddfca-2867-4e1a-8023-1f0b7c15c4bc admin admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:40:25.326972 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3ddfca-2867-4e1a-8023-1f0b7c15c4bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:40:25.327749 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3ddfca-2867-4e1a-8023-1f0b7c15c4bc admin admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:40:25.330647 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b3ddfca-2867-4e1a-8023-1f0b7c15c4bc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:40:25.336576 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b3ddfca-2867-4e1a-8023-1f0b7c15c4bc admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:40:25.337948 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 845/1679] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 1 16:40:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 16:40:25.349697 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d87bb3-6acc-4bf0-aeae-a6d4793dc15b None None] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:40:25.353321 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d87bb3-6acc-4bf0-aeae-a6d4793dc15b None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:40:25.353748 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d87bb3-6acc-4bf0-aeae-a6d4793dc15b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:40:25.370811 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 835/1680] 10.222.0.57 () {58 vars in 1121 bytes} [Fri Jul 1 16:40:24 2022] GET /identity/v3/roles => generated 1420 bytes in 377 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 16:40:25.375636 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3ddfca-2867-4e1a-8023-1f0b7c15c4bc admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:40:25.377527 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-3b3ddfca-2867-4e1a-8023-1f0b7c15c4bc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:40:25.429002 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3ddfca-2867-4e1a-8023-1f0b7c15c4bc admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:40:25.431413 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.auth.core [None req-75d87bb3-6acc-4bf0-aeae-a6d4793dc15b None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80312) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:40:25.436856 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-75d87bb3-6acc-4bf0-aeae-a6d4793dc15b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:40:25.445704 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91ff114-983d-43e9-a833-1f35f5c71ff3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:40:25.446916 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91ff114-983d-43e9-a833-1f35f5c71ff3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:40:25.447207 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91ff114-983d-43e9-a833-1f35f5c71ff3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:40:25.447529 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91ff114-983d-43e9-a833-1f35f5c71ff3 admin admin] PATH_INFO: `/v3/projects/53eba863d47f4e669ef367658a83a96b/users/15e12b51d19c451d91be7bcecc6b66a1/roles/84ee739146934c1ab32c501ff8093644` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:40:25.447826 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 846/1681] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 1 16:40:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 16:40:25.452306 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d87bb3-6acc-4bf0-aeae-a6d4793dc15b None None] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:40:25.457772 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b91ff114-983d-43e9-a833-1f35f5c71ff3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=53eba863d47f4e669ef367658a83a96b, user_id=15e12b51d19c451d91be7bcecc6b66a1, role_id=84ee739146934c1ab32c501ff8093644)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:40:25.460777 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b91ff114-983d-43e9-a833-1f35f5c71ff3 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:40:25.482278 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d87bb3-6acc-4bf0-aeae-a6d4793dc15b None None] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:40:25.496397 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 836/1682] 10.222.0.57 () {60 vars in 1482 bytes} [Fri Jul 1 16:40:25 2022] PUT /identity/v3/projects/53eba863d47f4e669ef367658a83a96b/users/15e12b51d19c451d91be7bcecc6b66a1/roles/84ee739146934c1ab32c501ff8093644 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 16:40:25.508508 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c808f29-769c-4442-bfbc-27b925fe04a8 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:40:25.508851 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c808f29-769c-4442-bfbc-27b925fe04a8 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:40:25.509183 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c808f29-769c-4442-bfbc-27b925fe04a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:40:25.550877 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-9c808f29-769c-4442-bfbc-27b925fe04a8 None None] MFA Rules not processed for user `15e12b51d19c451d91be7bcecc6b66a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:40:25.552251 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-9c808f29-769c-4442-bfbc-27b925fe04a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:40:25.617354 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:40:25.704079 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 837/1683] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 1 16:40:25 2022] POST /identity/v3/auth/tokens => generated 3343 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 16:40:25.713522 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c808f29-769c-4442-bfbc-27b925fe04a8 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:40:25.715440 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-9c808f29-769c-4442-bfbc-27b925fe04a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:40:25.724560 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e62da2-94e2-4b96-9ad2-17f932300b07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:40:25.726055 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e62da2-94e2-4b96-9ad2-17f932300b07 admin admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:40:25.727796 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e62da2-94e2-4b96-9ad2-17f932300b07 admin admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:40:25.728716 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e62da2-94e2-4b96-9ad2-17f932300b07 admin admin] PATH_INFO: `/v3/domains` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:40:25.732140 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e62da2-94e2-4b96-9ad2-17f932300b07 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80312) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:40:25.732631 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e62da2-94e2-4b96-9ad2-17f932300b07 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:40:25.750338 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e62da2-94e2-4b96-9ad2-17f932300b07 admin admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:40:25.793171 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 847/1684] 10.222.0.57 () {58 vars in 1152 bytes} [Fri Jul 1 16:40:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 341 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:40:25.794137 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e62da2-94e2-4b96-9ad2-17f932300b07 admin admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:40:25.812010 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e62da2-94e2-4b96-9ad2-17f932300b07 admin admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:40:25.849094 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751ee3f7-ea5f-40e0-95fe-a3eb8daaa3f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:40:25.849801 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751ee3f7-ea5f-40e0-95fe-a3eb8daaa3f8 admin admin] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:40:25.850085 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751ee3f7-ea5f-40e0-95fe-a3eb8daaa3f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:40:25.850408 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751ee3f7-ea5f-40e0-95fe-a3eb8daaa3f8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:40:25.851096 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-751ee3f7-ea5f-40e0-95fe-a3eb8daaa3f8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:40:25.853173 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-751ee3f7-ea5f-40e0-95fe-a3eb8daaa3f8 admin admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:40:25.884441 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c808f29-769c-4442-bfbc-27b925fe04a8 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:40:25.902398 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f70355e-b03a-4e94-a698-f3f071a39248 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbddf971a2354bbdbb9da1d63f3abf88', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0788e6725a84e57a4174284c1845755', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:40:25.902869 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f70355e-b03a-4e94-a698-f3f071a39248 service cinder] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:40:25.903244 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f70355e-b03a-4e94-a698-f3f071a39248 service cinder] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:40:25.903529 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f70355e-b03a-4e94-a698-f3f071a39248 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:40:25.921540 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 848/1685] 10.222.0.57 () {60 vars in 1152 bytes} [Fri Jul 1 16:40:25 2022] POST /identity/v3/projects => generated 365 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 16:40:25.927263 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751ee3f7-ea5f-40e0-95fe-a3eb8daaa3f8 admin admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:40:25.937379 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f70355e-b03a-4e94-a698-f3f071a39248 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:40:25.941946 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f70355e-b03a-4e94-a698-f3f071a39248 service cinder] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:40:25.982155 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 838/1686] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 1 16:40:25 2022] GET /identity/v3/auth/tokens => generated 3343 bytes in 268 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 16:40:26.000470 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=93,keystone:DELETE=5,keystone:INSERT=2 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:40:26.092445 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751ee3f7-ea5f-40e0-95fe-a3eb8daaa3f8 admin admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:40:26.147424 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966e0b74-3baa-4f72-b274-bad94e9baaaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:40:26.148676 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966e0b74-3baa-4f72-b274-bad94e9baaaa admin admin] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:40:26.149178 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966e0b74-3baa-4f72-b274-bad94e9baaaa admin admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:40:26.149667 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966e0b74-3baa-4f72-b274-bad94e9baaaa admin admin] PATH_INFO: `/v3/users` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:40:26.155148 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-966e0b74-3baa-4f72-b274-bad94e9baaaa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:40:26.157528 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-966e0b74-3baa-4f72-b274-bad94e9baaaa admin admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:40:26.196606 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 849/1687] 10.222.0.57 () {60 vars in 1143 bytes} [Fri Jul 1 16:40:25 2022] POST /identity/v3/users => generated 283 bytes in 270 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 16:40:26.200334 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f70355e-b03a-4e94-a698-f3f071a39248 service cinder] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:40:26.230520 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f70355e-b03a-4e94-a698-f3f071a39248 service cinder] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:40:26.248826 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4587c508-4f30-48b1-97e7-2a7b2f8ca349 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:40:26.251485 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4587c508-4f30-48b1-97e7-2a7b2f8ca349 admin admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:40:26.252794 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4587c508-4f30-48b1-97e7-2a7b2f8ca349 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:40:26.254497 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4587c508-4f30-48b1-97e7-2a7b2f8ca349 admin admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:40:26.256861 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4587c508-4f30-48b1-97e7-2a7b2f8ca349 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:40:26.258920 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4587c508-4f30-48b1-97e7-2a7b2f8ca349 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:40:26.308493 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 839/1688] 10.222.0.57 () {58 vars in 1121 bytes} [Fri Jul 1 16:40:26 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 16:40:26.310804 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966e0b74-3baa-4f72-b274-bad94e9baaaa admin admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:40:26.334480 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966e0b74-3baa-4f72-b274-bad94e9baaaa admin admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:40:26.356398 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a53de37-3bad-47a9-ad1a-5d1259bb7eee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:40:26.360039 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a53de37-3bad-47a9-ad1a-5d1259bb7eee admin admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:40:26.362883 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a53de37-3bad-47a9-ad1a-5d1259bb7eee admin admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:40:26.364556 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a53de37-3bad-47a9-ad1a-5d1259bb7eee admin admin] PATH_INFO: `/v3/roles` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:40:26.366016 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a53de37-3bad-47a9-ad1a-5d1259bb7eee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:40:26.373476 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a53de37-3bad-47a9-ad1a-5d1259bb7eee admin admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:40:26.403979 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 850/1689] 10.222.0.57 () {58 vars in 1121 bytes} [Fri Jul 1 16:40:26 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 16:40:26.405708 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4587c508-4f30-48b1-97e7-2a7b2f8ca349 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:40:26.424510 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4587c508-4f30-48b1-97e7-2a7b2f8ca349 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:40:26.447748 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a53de37-3bad-47a9-ad1a-5d1259bb7eee admin admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:40:26.449566 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-7a53de37-3bad-47a9-ad1a-5d1259bb7eee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:40:26.451932 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90994b60-433d-469f-a613-63b575ae5c0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:40:26.452852 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90994b60-433d-469f-a613-63b575ae5c0c admin admin] REQUEST_METHOD: `PUT` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:40:26.453141 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90994b60-433d-469f-a613-63b575ae5c0c admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:40:26.453420 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90994b60-433d-469f-a613-63b575ae5c0c admin admin] PATH_INFO: `/v3/projects/806275777ee4475c9bad3ac8af535f84/users/dbafca13ccd9453892842452941b2395/roles/983be40bdab04c478eaa90c6b6a46b14` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:40:26.529923 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90994b60-433d-469f-a613-63b575ae5c0c admin admin] RBAC: Authorizing `identity:create_grant(project_id=806275777ee4475c9bad3ac8af535f84, user_id=dbafca13ccd9453892842452941b2395, role_id=983be40bdab04c478eaa90c6b6a46b14)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:40:26.538540 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90994b60-433d-469f-a613-63b575ae5c0c admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:40:26.575950 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 840/1690] 10.222.0.57 () {60 vars in 1482 bytes} [Fri Jul 1 16:40:26 2022] PUT /identity/v3/projects/806275777ee4475c9bad3ac8af535f84/users/dbafca13ccd9453892842452941b2395/roles/983be40bdab04c478eaa90c6b6a46b14 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 16:40:26.591153 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ab62b7-732b-47b4-ba76-3b750fad4d0b None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:40:26.592391 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ab62b7-732b-47b4-ba76-3b750fad4d0b None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:40:26.592682 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ab62b7-732b-47b4-ba76-3b750fad4d0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:40:26.668272 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a53de37-3bad-47a9-ad1a-5d1259bb7eee admin admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:40:26.728028 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-34ab62b7-732b-47b4-ba76-3b750fad4d0b None None] MFA Rules not processed for user `dbafca13ccd9453892842452941b2395`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:40:26.736154 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-34ab62b7-732b-47b4-ba76-3b750fad4d0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:40:26.871370 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1821b9-7807-4e32-988b-1552d0ac433d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbddf971a2354bbdbb9da1d63f3abf88', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0788e6725a84e57a4174284c1845755', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:40:26.872037 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1821b9-7807-4e32-988b-1552d0ac433d service cinder] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:40:26.872319 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1821b9-7807-4e32-988b-1552d0ac433d service cinder] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:40:26.872651 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1821b9-7807-4e32-988b-1552d0ac433d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:40:26.878475 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-7c1821b9-7807-4e32-988b-1552d0ac433d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:40:26.942766 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 841/1691] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 1 16:40:26 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 353 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 16:40:26.944274 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c41675-93ac-463f-8f52-49edbf39f37c None None] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:40:26.944567 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c41675-93ac-463f-8f52-49edbf39f37c None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:40:26.944846 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c41675-93ac-463f-8f52-49edbf39f37c None None] PATH_INFO: `/` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:40:26.961382 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 842/1692] 10.222.0.57 () {54 vars in 921 bytes} [Fri Jul 1 16:40:26 2022] GET /identity => generated 270 bytes in 18 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 16:40:26.962624 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c41675-93ac-463f-8f52-49edbf39f37c None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:40:27.027066 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:40:27.132439 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c1821b9-7807-4e32-988b-1552d0ac433d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:40:27.134758 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c1821b9-7807-4e32-988b-1552d0ac433d service cinder] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:40:27.173372 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 851/1693] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 1 16:40:26 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 726 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 01 16:40:27.173859 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c41675-93ac-463f-8f52-49edbf39f37c None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:40:27.175386 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f5f8f8-dacd-463c-b329-1ad0a8b917cc None None] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:40:27.176177 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f5f8f8-dacd-463c-b329-1ad0a8b917cc None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:40:27.176672 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f5f8f8-dacd-463c-b329-1ad0a8b917cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:40:27.204192 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.auth.core [None req-56f5f8f8-dacd-463c-b329-1ad0a8b917cc None None] MFA Rules not processed for user `bbc38af18dec4e9b8143df23af6aad43`. Rule list: `[]` (Enabled: `True`). {{(pid=80312) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:40:27.205877 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-56f5f8f8-dacd-463c-b329-1ad0a8b917cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:40:27.207539 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23020f5a-80ba-4741-a612-a0ba9dbecefc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:40:27.208442 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23020f5a-80ba-4741-a612-a0ba9dbecefc admin admin] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:40:27.208730 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23020f5a-80ba-4741-a612-a0ba9dbecefc admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:40:27.209263 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23020f5a-80ba-4741-a612-a0ba9dbecefc admin admin] PATH_INFO: `/v3/projects` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:40:27.210221 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23020f5a-80ba-4741-a612-a0ba9dbecefc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:40:27.212177 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23020f5a-80ba-4741-a612-a0ba9dbecefc admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:40:27.220799 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:40:27.294812 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 843/1694] 10.222.0.57 () {60 vars in 1152 bytes} [Fri Jul 1 16:40:26 2022] POST /identity/v3/projects => generated 365 bytes in 332 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 16:40:27.299603 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23020f5a-80ba-4741-a612-a0ba9dbecefc admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:40:27.426612 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 852/1695] 10.222.0.57 () {58 vars in 1018 bytes} [Fri Jul 1 16:40:27 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 252 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 16:40:27.463307 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23020f5a-80ba-4741-a612-a0ba9dbecefc admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:40:27.476601 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f57168c-2f65-43e4-b4ce-b1bb8f1cce1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:40:27.477278 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f57168c-2f65-43e4-b4ce-b1bb8f1cce1d admin admin] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:40:27.478488 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f57168c-2f65-43e4-b4ce-b1bb8f1cce1d admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:40:27.478975 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f57168c-2f65-43e4-b4ce-b1bb8f1cce1d admin admin] PATH_INFO: `/v3/users` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:40:27.480817 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f57168c-2f65-43e4-b4ce-b1bb8f1cce1d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:40:27.484443 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f57168c-2f65-43e4-b4ce-b1bb8f1cce1d admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:40:27.494783 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebe7510-b021-4c21-a601-7ce632b0a601 None None] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:40:27.495001 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebe7510-b021-4c21-a601-7ce632b0a601 None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:40:27.495462 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebe7510-b021-4c21-a601-7ce632b0a601 None None] PATH_INFO: `/` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:40:27.497150 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 853/1696] 10.222.0.57 () {54 vars in 960 bytes} [Fri Jul 1 16:40:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 01 16:40:27.502814 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f8a07c-849c-459d-9535-264d9c07f7b4 None None] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:40:27.503841 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f8a07c-849c-459d-9535-264d9c07f7b4 None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:40:27.504955 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f8a07c-849c-459d-9535-264d9c07f7b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:40:27.540991 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.auth.core [None req-a2f8a07c-849c-459d-9535-264d9c07f7b4 None None] MFA Rules not processed for user `bbc38af18dec4e9b8143df23af6aad43`. Rule list: `[]` (Enabled: `True`). {{(pid=80312) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:40:27.542239 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-a2f8a07c-849c-459d-9535-264d9c07f7b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:40:27.548448 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 844/1697] 10.222.0.57 () {60 vars in 1143 bytes} [Fri Jul 1 16:40:27 2022] POST /identity/v3/users => generated 289 bytes in 249 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 16:40:27.549544 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f57168c-2f65-43e4-b4ce-b1bb8f1cce1d admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:40:27.564646 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f57168c-2f65-43e4-b4ce-b1bb8f1cce1d admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:40:27.576011 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778167ff-5a8b-4ff6-9e6e-dd6820c4a66b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:40:27.576789 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778167ff-5a8b-4ff6-9e6e-dd6820c4a66b admin admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:40:27.577141 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778167ff-5a8b-4ff6-9e6e-dd6820c4a66b admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:40:27.577537 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778167ff-5a8b-4ff6-9e6e-dd6820c4a66b admin admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:40:27.578382 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-778167ff-5a8b-4ff6-9e6e-dd6820c4a66b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:40:27.582946 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-778167ff-5a8b-4ff6-9e6e-dd6820c4a66b admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:40:27.612990 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 845/1698] 10.222.0.57 () {58 vars in 1121 bytes} [Fri Jul 1 16:40:27 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 16:40:27.614796 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778167ff-5a8b-4ff6-9e6e-dd6820c4a66b admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:40:27.633653 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778167ff-5a8b-4ff6-9e6e-dd6820c4a66b admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:40:27.643807 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1845a5da-5e80-47e0-9107-52b4440c87ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:40:27.645681 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1845a5da-5e80-47e0-9107-52b4440c87ef admin admin] REQUEST_METHOD: `PUT` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:40:27.646629 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1845a5da-5e80-47e0-9107-52b4440c87ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:40:27.647079 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1845a5da-5e80-47e0-9107-52b4440c87ef admin admin] PATH_INFO: `/v3/projects/cfd794279b654d3c990f6d84f4dad747/users/10bbd9ddbcec4299b8635d6df0a41a4d/roles/84ee739146934c1ab32c501ff8093644` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:40:27.671996 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1845a5da-5e80-47e0-9107-52b4440c87ef admin admin] RBAC: Authorizing `identity:create_grant(project_id=cfd794279b654d3c990f6d84f4dad747, user_id=10bbd9ddbcec4299b8635d6df0a41a4d, role_id=84ee739146934c1ab32c501ff8093644)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:40:27.680168 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1845a5da-5e80-47e0-9107-52b4440c87ef admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:40:27.699122 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 854/1699] 10.222.0.57 () {58 vars in 1057 bytes} [Fri Jul 1 16:40:27 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 16:40:27.707221 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f8a07c-849c-459d-9535-264d9c07f7b4 None None] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:40:27.711616 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 846/1700] 10.222.0.57 () {60 vars in 1482 bytes} [Fri Jul 1 16:40:27 2022] PUT /identity/v3/projects/cfd794279b654d3c990f6d84f4dad747/users/10bbd9ddbcec4299b8635d6df0a41a4d/roles/84ee739146934c1ab32c501ff8093644 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 16:40:27.715420 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1845a5da-5e80-47e0-9107-52b4440c87ef admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:40:27.731923 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f8a07c-849c-459d-9535-264d9c07f7b4 None None] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:40:27.863855 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1845a5da-5e80-47e0-9107-52b4440c87ef admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:40:27.871127 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c51d18-b81e-48ac-808c-50b3621702a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:40:27.872373 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c51d18-b81e-48ac-808c-50b3621702a6 admin admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:40:27.873860 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c51d18-b81e-48ac-808c-50b3621702a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:40:27.874426 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c51d18-b81e-48ac-808c-50b3621702a6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:40:27.875439 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c51d18-b81e-48ac-808c-50b3621702a6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:40:27.877599 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c51d18-b81e-48ac-808c-50b3621702a6 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:40:27.880464 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:40:27.894260 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 847/1701] 10.222.0.57 () {58 vars in 1121 bytes} [Fri Jul 1 16:40:27 2022] GET /identity/v3/roles => generated 1420 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 16:40:27.901367 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c51d18-b81e-48ac-808c-50b3621702a6 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:40:27.902245 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4390aa34-1525-4493-89b9-31b063aa945e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc38af18dec4e9b8143df23af6aad43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0788e6725a84e57a4174284c1845755', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:40:27.903048 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4390aa34-1525-4493-89b9-31b063aa945e service placement] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:40:27.903508 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4390aa34-1525-4493-89b9-31b063aa945e service placement] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:40:27.903825 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4390aa34-1525-4493-89b9-31b063aa945e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:40:27.924727 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c51d18-b81e-48ac-808c-50b3621702a6 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:40:27.925119 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4390aa34-1525-4493-89b9-31b063aa945e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:40:27.926805 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4390aa34-1525-4493-89b9-31b063aa945e service placement] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:40:27.944479 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac0ad06-59a8-4309-a7c9-5f7560c29604 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:40:27.945780 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac0ad06-59a8-4309-a7c9-5f7560c29604 admin admin] REQUEST_METHOD: `PUT` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:40:27.945780 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac0ad06-59a8-4309-a7c9-5f7560c29604 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:40:27.945941 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac0ad06-59a8-4309-a7c9-5f7560c29604 admin admin] PATH_INFO: `/v3/projects/cfd794279b654d3c990f6d84f4dad747/users/10bbd9ddbcec4299b8635d6df0a41a4d/roles/84ee739146934c1ab32c501ff8093644` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:40:27.950440 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac0ad06-59a8-4309-a7c9-5f7560c29604 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cfd794279b654d3c990f6d84f4dad747, user_id=10bbd9ddbcec4299b8635d6df0a41a4d, role_id=84ee739146934c1ab32c501ff8093644)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:40:27.962384 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac0ad06-59a8-4309-a7c9-5f7560c29604 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:40:27.963813 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 855/1702] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 1 16:40:27 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 257 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 16:40:28.008229 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 848/1703] 10.222.0.57 () {60 vars in 1482 bytes} [Fri Jul 1 16:40:27 2022] PUT /identity/v3/projects/cfd794279b654d3c990f6d84f4dad747/users/10bbd9ddbcec4299b8635d6df0a41a4d/roles/84ee739146934c1ab32c501ff8093644 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 16:40:28.010778 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2543e3ce-cdc6-4e30-8004-3b207ed1266c None None] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:40:28.010778 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2543e3ce-cdc6-4e30-8004-3b207ed1266c None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:40:28.010778 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2543e3ce-cdc6-4e30-8004-3b207ed1266c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:40:28.074393 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.auth.core [None req-2543e3ce-cdc6-4e30-8004-3b207ed1266c None None] MFA Rules not processed for user `10bbd9ddbcec4299b8635d6df0a41a4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80312) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:40:28.085622 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-2543e3ce-cdc6-4e30-8004-3b207ed1266c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:40:28.154536 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=100 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:40:28.191716 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac0ad06-59a8-4309-a7c9-5f7560c29604 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:40:28.194865 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-eac0ad06-59a8-4309-a7c9-5f7560c29604 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:40:28.275357 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 856/1704] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 1 16:40:28 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 268 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 16:40:28.291163 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2543e3ce-cdc6-4e30-8004-3b207ed1266c None None] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:40:28.385012 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac0ad06-59a8-4309-a7c9-5f7560c29604 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:40:28.390709 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327efdde-af50-4629-808a-138e75e75a69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:40:28.391668 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327efdde-af50-4629-808a-138e75e75a69 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:40:28.391954 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327efdde-af50-4629-808a-138e75e75a69 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:40:28.392229 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327efdde-af50-4629-808a-138e75e75a69 admin admin] PATH_INFO: `/v3/users/aca6b4b731ed4347827748e05fb284ea` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:40:28.394871 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-327efdde-af50-4629-808a-138e75e75a69 admin admin] RBAC: Authorizing `identity:delete_user(user_id=aca6b4b731ed4347827748e05fb284ea)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:40:28.397312 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-327efdde-af50-4629-808a-138e75e75a69 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:40:28.435648 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2543e3ce-cdc6-4e30-8004-3b207ed1266c None None] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:40:28.445881 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965f95e7-6cde-4829-b397-548e7704c47e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbddf971a2354bbdbb9da1d63f3abf88', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0788e6725a84e57a4174284c1845755', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:40:28.446908 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965f95e7-6cde-4829-b397-548e7704c47e service cinder] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:40:28.447298 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965f95e7-6cde-4829-b397-548e7704c47e service cinder] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:40:28.447703 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965f95e7-6cde-4829-b397-548e7704c47e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:40:28.467010 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-327efdde-af50-4629-808a-138e75e75a69 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aca6b4b731ed4347827748e05fb284ea'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:40:28.467801 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-327efdde-af50-4629-808a-138e75e75a69 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aca6b4b731ed4347827748e05fb284ea'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:40:28.468502 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-327efdde-af50-4629-808a-138e75e75a69 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aca6b4b731ed4347827748e05fb284ea'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:40:28.468844 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-965f95e7-6cde-4829-b397-548e7704c47e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:40:28.470923 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-965f95e7-6cde-4829-b397-548e7704c47e service cinder] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:40:28.479755 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-327efdde-af50-4629-808a-138e75e75a69 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aca6b4b731ed4347827748e05fb284ea'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:40:28.500996 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-327efdde-af50-4629-808a-138e75e75a69 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aca6b4b731ed4347827748e05fb284ea'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:40:28.508094 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 857/1705] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 1 16:40:28 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 218 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 01 16:40:28.519494 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 849/1706] 10.222.0.57 () {58 vars in 1223 bytes} [Fri Jul 1 16:40:28 2022] DELETE /identity/v3/users/aca6b4b731ed4347827748e05fb284ea => generated 0 bytes in 328 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 01 16:40:28.525048 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3ce77e-8985-4e1d-bebc-c028a5200b48 None None] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:40:28.525897 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3ce77e-8985-4e1d-bebc-c028a5200b48 None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:40:28.526647 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3ce77e-8985-4e1d-bebc-c028a5200b48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:40:28.554105 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.auth.core [None req-1a3ce77e-8985-4e1d-bebc-c028a5200b48 None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80312) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:40:28.555738 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-1a3ce77e-8985-4e1d-bebc-c028a5200b48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:40:28.740762 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 858/1707] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 1 16:40:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 16:40:28.775512 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327efdde-af50-4629-808a-138e75e75a69 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:40:28.777938 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-327efdde-af50-4629-808a-138e75e75a69 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:40:28.902589 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327efdde-af50-4629-808a-138e75e75a69 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:40:28.913260 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca24cc0-5e76-40ae-817b-bfaa86e05396 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ed200ce29684c5f80354fa9a8638fe2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0788e6725a84e57a4174284c1845755', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:40:28.914095 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca24cc0-5e76-40ae-817b-bfaa86e05396 service neutron] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:40:28.914565 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca24cc0-5e76-40ae-817b-bfaa86e05396 service neutron] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:40:28.914955 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca24cc0-5e76-40ae-817b-bfaa86e05396 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:40:28.945953 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca24cc0-5e76-40ae-817b-bfaa86e05396 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:40:28.952047 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca24cc0-5e76-40ae-817b-bfaa86e05396 service neutron] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:40:28.979469 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 850/1708] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 1 16:40:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 204 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 16:40:29.467080 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3ce77e-8985-4e1d-bebc-c028a5200b48 None None] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:40:29.469290 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-1a3ce77e-8985-4e1d-bebc-c028a5200b48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:40:29.490952 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3ce77e-8985-4e1d-bebc-c028a5200b48 None None] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:40:29.504174 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c253cfb-678c-438f-9cb3-13525521e882 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:40:29.505317 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c253cfb-678c-438f-9cb3-13525521e882 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:40:29.505711 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c253cfb-678c-438f-9cb3-13525521e882 admin admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:40:29.506374 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c253cfb-678c-438f-9cb3-13525521e882 admin admin] PATH_INFO: `/v3/users/dbafca13ccd9453892842452941b2395` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:40:29.509409 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c253cfb-678c-438f-9cb3-13525521e882 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dbafca13ccd9453892842452941b2395)` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:40:29.511701 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c253cfb-678c-438f-9cb3-13525521e882 admin admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:40:29.630863 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-1c253cfb-678c-438f-9cb3-13525521e882 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dbafca13ccd9453892842452941b2395'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:40:29.649159 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=92,keystone:DELETE=8 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:40:29.660928 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-1c253cfb-678c-438f-9cb3-13525521e882 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dbafca13ccd9453892842452941b2395'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:40:29.686805 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-1c253cfb-678c-438f-9cb3-13525521e882 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dbafca13ccd9453892842452941b2395'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:40:29.688075 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-1c253cfb-678c-438f-9cb3-13525521e882 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dbafca13ccd9453892842452941b2395'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:40:29.688993 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-1c253cfb-678c-438f-9cb3-13525521e882 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dbafca13ccd9453892842452941b2395'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:40:29.716083 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 859/1709] 10.222.0.57 () {58 vars in 1223 bytes} [Fri Jul 1 16:40:29 2022] DELETE /identity/v3/users/dbafca13ccd9453892842452941b2395 => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 16:40:29.727438 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcc1023-d120-4c57-a788-188bbe315b5a None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:40:29.728159 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcc1023-d120-4c57-a788-188bbe315b5a None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:40:29.728562 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcc1023-d120-4c57-a788-188bbe315b5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:40:29.737065 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:40:29.767563 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-9dcc1023-d120-4c57-a788-188bbe315b5a None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:40:29.771526 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-9dcc1023-d120-4c57-a788-188bbe315b5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:40:29.934417 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 851/1710] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 1 16:40:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 16:40:29.981151 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c253cfb-678c-438f-9cb3-13525521e882 admin admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:40:29.982846 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-1c253cfb-678c-438f-9cb3-13525521e882 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:40:30.080428 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c253cfb-678c-438f-9cb3-13525521e882 admin admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:40:30.089477 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6207772c-f9c4-4001-b334-e3e3c2d51643 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ed200ce29684c5f80354fa9a8638fe2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0788e6725a84e57a4174284c1845755', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:40:30.090187 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6207772c-f9c4-4001-b334-e3e3c2d51643 service neutron] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:40:30.090744 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6207772c-f9c4-4001-b334-e3e3c2d51643 service neutron] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:40:30.090854 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6207772c-f9c4-4001-b334-e3e3c2d51643 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:40:30.125470 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6207772c-f9c4-4001-b334-e3e3c2d51643 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:40:30.128235 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6207772c-f9c4-4001-b334-e3e3c2d51643 service neutron] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:40:30.161858 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 860/1711] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 1 16:40:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 181 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 16:40:30.218798 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf63389-5dd7-47d6-8ee8-c74fce12ebb9 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:40:30.219276 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf63389-5dd7-47d6-8ee8-c74fce12ebb9 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:40:30.219704 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf63389-5dd7-47d6-8ee8-c74fce12ebb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:40:30.253445 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-0cf63389-5dd7-47d6-8ee8-c74fce12ebb9 None None] MFA Rules not processed for user `7c4da21fbfa3472cac6e766a2f4799d8`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:40:30.255779 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-0cf63389-5dd7-47d6-8ee8-c74fce12ebb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:40:30.388687 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 852/1712] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 1 16:40:30 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 16:40:30.544140 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b733e873-38b6-40e2-a217-bb9f41cd546f None None] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:40:30.544825 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b733e873-38b6-40e2-a217-bb9f41cd546f None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:40:30.544825 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b733e873-38b6-40e2-a217-bb9f41cd546f None None] PATH_INFO: `/` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:40:30.548032 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 861/1713] 10.222.0.57 () {54 vars in 921 bytes} [Fri Jul 1 16:40:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 16:40:30.700976 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4322a083-f55f-4834-9818-19e4b90b4fb0 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:40:30.701299 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4322a083-f55f-4834-9818-19e4b90b4fb0 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:40:30.701638 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4322a083-f55f-4834-9818-19e4b90b4fb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:40:30.735753 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-4322a083-f55f-4834-9818-19e4b90b4fb0 None None] MFA Rules not processed for user `bbc38af18dec4e9b8143df23af6aad43`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:40:30.738718 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-4322a083-f55f-4834-9818-19e4b90b4fb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:40:30.889146 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 853/1714] 10.222.0.57 () {58 vars in 1018 bytes} [Fri Jul 1 16:40:30 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 16:40:30.908437 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b733e873-38b6-40e2-a217-bb9f41cd546f None None] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:40:30.911417 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-b733e873-38b6-40e2-a217-bb9f41cd546f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:40:30.953055 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b733e873-38b6-40e2-a217-bb9f41cd546f None None] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:40:30.962839 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4322a083-f55f-4834-9818-19e4b90b4fb0 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:40:30.966468 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-4322a083-f55f-4834-9818-19e4b90b4fb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:40:30.968156 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba97ec4-7d9e-4a6f-8708-5956ebad12cb service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc38af18dec4e9b8143df23af6aad43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0788e6725a84e57a4174284c1845755', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:40:30.969539 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba97ec4-7d9e-4a6f-8708-5956ebad12cb service placement] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:40:30.969971 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba97ec4-7d9e-4a6f-8708-5956ebad12cb service placement] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:40:30.970422 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba97ec4-7d9e-4a6f-8708-5956ebad12cb service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:40:30.993469 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba97ec4-7d9e-4a6f-8708-5956ebad12cb service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:40:30.995584 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba97ec4-7d9e-4a6f-8708-5956ebad12cb service placement] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:40:31.018247 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 862/1715] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 1 16:40:30 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 16:40:31.023360 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4322a083-f55f-4834-9818-19e4b90b4fb0 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:40:31.043870 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cedaa132-c8d7-4d50-b157-db5b02da5282 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:40:31.044563 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedaa132-c8d7-4d50-b157-db5b02da5282 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:40:31.044979 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedaa132-c8d7-4d50-b157-db5b02da5282 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:40:31.045275 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedaa132-c8d7-4d50-b157-db5b02da5282 admin admin] PATH_INFO: `/v3/users/f5ce7822f6b2469cba93544928f7957f` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:40:31.051941 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cedaa132-c8d7-4d50-b157-db5b02da5282 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f5ce7822f6b2469cba93544928f7957f)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:40:31.054336 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cedaa132-c8d7-4d50-b157-db5b02da5282 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:40:31.125953 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba97ec4-7d9e-4a6f-8708-5956ebad12cb service placement] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:40:31.127264 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-9ba97ec4-7d9e-4a6f-8708-5956ebad12cb service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:40:31.135843 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:40:31.148129 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-cedaa132-c8d7-4d50-b157-db5b02da5282 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f5ce7822f6b2469cba93544928f7957f'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:40:31.149223 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-cedaa132-c8d7-4d50-b157-db5b02da5282 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f5ce7822f6b2469cba93544928f7957f'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:40:31.149743 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-cedaa132-c8d7-4d50-b157-db5b02da5282 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f5ce7822f6b2469cba93544928f7957f'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:40:31.160653 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba97ec4-7d9e-4a6f-8708-5956ebad12cb service placement] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:40:31.173589 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79517f6e-8264-400a-8a61-3d7e3b20d28a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:40:31.175912 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79517f6e-8264-400a-8a61-3d7e3b20d28a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:40:31.176383 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79517f6e-8264-400a-8a61-3d7e3b20d28a admin admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:40:31.176861 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-cedaa132-c8d7-4d50-b157-db5b02da5282 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f5ce7822f6b2469cba93544928f7957f'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:40:31.177496 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79517f6e-8264-400a-8a61-3d7e3b20d28a admin admin] PATH_INFO: `/v3/projects/eec1d42b049c4f8080d138d248280cac` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:40:31.188089 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79517f6e-8264-400a-8a61-3d7e3b20d28a admin admin] RBAC: Authorizing `identity:delete_project(project_id=eec1d42b049c4f8080d138d248280cac)` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:40:31.193634 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79517f6e-8264-400a-8a61-3d7e3b20d28a admin admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:40:31.196830 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-cedaa132-c8d7-4d50-b157-db5b02da5282 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f5ce7822f6b2469cba93544928f7957f'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:40:31.225563 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 854/1716] 10.222.0.57 () {58 vars in 1223 bytes} [Fri Jul 1 16:40:30 2022] DELETE /identity/v3/users/f5ce7822f6b2469cba93544928f7957f => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 01 16:40:31.336609 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-79517f6e-8264-400a-8a61-3d7e3b20d28a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eec1d42b049c4f8080d138d248280cac', 'request_id': 'req-79517f6e-8264-400a-8a61-3d7e3b20d28a'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:40:31.350622 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-79517f6e-8264-400a-8a61-3d7e3b20d28a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eec1d42b049c4f8080d138d248280cac', 'request_id': 'req-79517f6e-8264-400a-8a61-3d7e3b20d28a'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:40:31.370834 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-79517f6e-8264-400a-8a61-3d7e3b20d28a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eec1d42b049c4f8080d138d248280cac', 'request_id': 'req-79517f6e-8264-400a-8a61-3d7e3b20d28a'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:40:31.387062 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-79517f6e-8264-400a-8a61-3d7e3b20d28a admin admin] The token cache is being invalidate because project eec1d42b049c4f8080d138d248280cac was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80312) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 01 16:40:31.387441 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-79517f6e-8264-400a-8a61-3d7e3b20d28a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:40:31.388230 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-79517f6e-8264-400a-8a61-3d7e3b20d28a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:40:31.395457 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 863/1717] 10.222.0.57 () {58 vars in 1232 bytes} [Fri Jul 1 16:40:31 2022] DELETE /identity/v3/projects/eec1d42b049c4f8080d138d248280cac => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 01 16:40:31.403138 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2c5d51-f2f7-4b22-ab8e-cc28de3cd52c None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:40:31.404155 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2c5d51-f2f7-4b22-ab8e-cc28de3cd52c None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:40:31.405170 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2c5d51-f2f7-4b22-ab8e-cc28de3cd52c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:40:31.443896 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-4b2c5d51-f2f7-4b22-ab8e-cc28de3cd52c None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:40:31.445455 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-4b2c5d51-f2f7-4b22-ab8e-cc28de3cd52c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:40:31.610239 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 855/1718] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 1 16:40:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 16:40:31.615109 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203fb417-3fce-4abb-b23c-a94b42a44fc9 None None] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:40:31.615602 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203fb417-3fce-4abb-b23c-a94b42a44fc9 None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:40:31.615970 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203fb417-3fce-4abb-b23c-a94b42a44fc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:40:31.692774 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.auth.core [None req-203fb417-3fce-4abb-b23c-a94b42a44fc9 None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80312) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:40:31.694179 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-203fb417-3fce-4abb-b23c-a94b42a44fc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:40:31.730498 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 864/1719] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 1 16:40:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 16:40:31.734480 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2c5d51-f2f7-4b22-ab8e-cc28de3cd52c None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:40:31.762939 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2c5d51-f2f7-4b22-ab8e-cc28de3cd52c None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:40:31.777519 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe842df0-4d7a-4e0c-9b8e-0ae6729835af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:40:31.779392 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe842df0-4d7a-4e0c-9b8e-0ae6729835af admin admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:40:31.780538 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe842df0-4d7a-4e0c-9b8e-0ae6729835af admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:40:31.781141 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe842df0-4d7a-4e0c-9b8e-0ae6729835af admin admin] PATH_INFO: `/v3/domains` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:40:31.785235 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe842df0-4d7a-4e0c-9b8e-0ae6729835af admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:40:31.785921 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe842df0-4d7a-4e0c-9b8e-0ae6729835af admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:40:31.789751 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe842df0-4d7a-4e0c-9b8e-0ae6729835af admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:40:31.822944 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 856/1720] 10.222.0.57 () {58 vars in 1152 bytes} [Fri Jul 1 16:40:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:40:31.827255 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203fb417-3fce-4abb-b23c-a94b42a44fc9 None None] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:40:31.846026 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203fb417-3fce-4abb-b23c-a94b42a44fc9 None None] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:40:31.856072 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61fe180a-0a72-4a68-a3cd-95c3e76a3539 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:40:31.856764 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fe180a-0a72-4a68-a3cd-95c3e76a3539 admin admin] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:40:31.857835 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fe180a-0a72-4a68-a3cd-95c3e76a3539 admin admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:40:31.857835 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fe180a-0a72-4a68-a3cd-95c3e76a3539 admin admin] PATH_INFO: `/v3/projects` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:40:31.881055 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61fe180a-0a72-4a68-a3cd-95c3e76a3539 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:40:31.883214 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61fe180a-0a72-4a68-a3cd-95c3e76a3539 admin admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:40:31.947888 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 865/1721] 10.222.0.57 () {60 vars in 1152 bytes} [Fri Jul 1 16:40:31 2022] POST /identity/v3/projects => generated 373 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 16:40:31.954487 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe842df0-4d7a-4e0c-9b8e-0ae6729835af admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:40:31.970691 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe842df0-4d7a-4e0c-9b8e-0ae6729835af admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:40:31.999761 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85eaec99-685b-4416-bd8d-7931790eddb0 None None] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:40:31.999964 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85eaec99-685b-4416-bd8d-7931790eddb0 None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:40:32.000241 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85eaec99-685b-4416-bd8d-7931790eddb0 None None] PATH_INFO: `/` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:40:32.002057 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 866/1722] 10.222.0.57 () {54 vars in 912 bytes} [Fri Jul 1 16:40:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 01 16:40:32.009974 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7a8c34-89ec-43b9-8e73-a05b0b0aa920 None None] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:40:32.010265 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7a8c34-89ec-43b9-8e73-a05b0b0aa920 None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:40:32.010584 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7a8c34-89ec-43b9-8e73-a05b0b0aa920 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:40:32.090933 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=88,keystone:DELETE=9,keystone:INSERT=3 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:40:32.105497 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.auth.core [None req-ce7a8c34-89ec-43b9-8e73-a05b0b0aa920 None None] MFA Rules not processed for user `4ed200ce29684c5f80354fa9a8638fe2`. Rule list: `[]` (Enabled: `True`). {{(pid=80312) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:40:32.115859 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-ce7a8c34-89ec-43b9-8e73-a05b0b0aa920 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:40:32.125710 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2f156a-eed8-402d-a025-84dd95fa4897 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:40:32.126961 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2f156a-eed8-402d-a025-84dd95fa4897 admin admin] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:40:32.134225 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2f156a-eed8-402d-a025-84dd95fa4897 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:40:32.134698 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2f156a-eed8-402d-a025-84dd95fa4897 admin admin] PATH_INFO: `/v3/users` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:40:32.136047 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef2f156a-eed8-402d-a025-84dd95fa4897 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:40:32.139408 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef2f156a-eed8-402d-a025-84dd95fa4897 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:40:32.198584 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 857/1723] 10.222.0.57 () {60 vars in 1143 bytes} [Fri Jul 1 16:40:31 2022] POST /identity/v3/users => generated 287 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 16:40:32.199524 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2f156a-eed8-402d-a025-84dd95fa4897 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:40:32.203268 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-ef2f156a-eed8-402d-a025-84dd95fa4897 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:40:32.231698 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2f156a-eed8-402d-a025-84dd95fa4897 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:40:32.245753 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e5aa0b-8b31-4d79-99b9-90aa4edb03ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:40:32.248669 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e5aa0b-8b31-4d79-99b9-90aa4edb03ff admin admin] REQUEST_METHOD: `DELETE` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:40:32.253931 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e5aa0b-8b31-4d79-99b9-90aa4edb03ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:40:32.255131 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e5aa0b-8b31-4d79-99b9-90aa4edb03ff admin admin] PATH_INFO: `/v3/projects/806275777ee4475c9bad3ac8af535f84` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:40:32.260067 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e5aa0b-8b31-4d79-99b9-90aa4edb03ff admin admin] RBAC: Authorizing `identity:delete_project(project_id=806275777ee4475c9bad3ac8af535f84)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:40:32.263821 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e5aa0b-8b31-4d79-99b9-90aa4edb03ff admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:40:32.268948 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 867/1724] 10.222.0.57 () {58 vars in 1009 bytes} [Fri Jul 1 16:40:32 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 259 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 01 16:40:32.274509 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7a8c34-89ec-43b9-8e73-a05b0b0aa920 None None] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:40:32.308044 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7a8c34-89ec-43b9-8e73-a05b0b0aa920 None None] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:40:32.316929 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5564deb4-217a-4445-931a-48257826b722 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:40:32.318673 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5564deb4-217a-4445-931a-48257826b722 admin admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:40:32.321540 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5564deb4-217a-4445-931a-48257826b722 admin admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:40:32.322262 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5564deb4-217a-4445-931a-48257826b722 admin admin] PATH_INFO: `/v3/roles` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:40:32.323132 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5564deb4-217a-4445-931a-48257826b722 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:40:32.327033 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5564deb4-217a-4445-931a-48257826b722 admin admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:40:32.351957 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 868/1725] 10.222.0.57 () {58 vars in 1121 bytes} [Fri Jul 1 16:40:32 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 16:40:32.352871 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5564deb4-217a-4445-931a-48257826b722 admin admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:40:32.356544 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=90,keystone:DELETE=6,keystone:INSERT=4 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:40:32.361415 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-5564deb4-217a-4445-931a-48257826b722 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:40:32.391261 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-c8e5aa0b-8b31-4d79-99b9-90aa4edb03ff admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '806275777ee4475c9bad3ac8af535f84', 'request_id': 'req-c8e5aa0b-8b31-4d79-99b9-90aa4edb03ff'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:40:32.394951 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5564deb4-217a-4445-931a-48257826b722 admin admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:40:32.403837 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-c8e5aa0b-8b31-4d79-99b9-90aa4edb03ff admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '806275777ee4475c9bad3ac8af535f84', 'request_id': 'req-c8e5aa0b-8b31-4d79-99b9-90aa4edb03ff'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:40:32.427169 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-c8e5aa0b-8b31-4d79-99b9-90aa4edb03ff admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '806275777ee4475c9bad3ac8af535f84', 'request_id': 'req-c8e5aa0b-8b31-4d79-99b9-90aa4edb03ff'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:40:32.474526 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-c8e5aa0b-8b31-4d79-99b9-90aa4edb03ff admin admin] The token cache is being invalidate because project 806275777ee4475c9bad3ac8af535f84 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80311) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 01 16:40:32.475069 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-c8e5aa0b-8b31-4d79-99b9-90aa4edb03ff admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:40:32.475940 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-c8e5aa0b-8b31-4d79-99b9-90aa4edb03ff admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:40:32.478120 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 858/1726] 10.222.0.57 () {58 vars in 1232 bytes} [Fri Jul 1 16:40:32 2022] DELETE /identity/v3/projects/806275777ee4475c9bad3ac8af535f84 => generated 0 bytes in 279 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 16:40:32.481070 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e5aa0b-8b31-4d79-99b9-90aa4edb03ff admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:40:32.482212 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-c8e5aa0b-8b31-4d79-99b9-90aa4edb03ff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:40:32.545978 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4fc07db-74eb-4e07-92ac-f0b39775ee8f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ed200ce29684c5f80354fa9a8638fe2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0788e6725a84e57a4174284c1845755', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:40:32.546871 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fc07db-74eb-4e07-92ac-f0b39775ee8f service neutron] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:40:32.547251 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fc07db-74eb-4e07-92ac-f0b39775ee8f service neutron] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:40:32.547569 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fc07db-74eb-4e07-92ac-f0b39775ee8f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:40:32.556280 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-e4fc07db-74eb-4e07-92ac-f0b39775ee8f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:40:32.580310 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4fc07db-74eb-4e07-92ac-f0b39775ee8f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:40:32.582204 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4fc07db-74eb-4e07-92ac-f0b39775ee8f service neutron] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:40:32.610300 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 869/1727] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 1 16:40:32 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 258 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 16:40:32.619547 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4fc07db-74eb-4e07-92ac-f0b39775ee8f service neutron] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:40:32.623929 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-e4fc07db-74eb-4e07-92ac-f0b39775ee8f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:40:32.691020 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e5aa0b-8b31-4d79-99b9-90aa4edb03ff admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:40:32.696375 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953d47c7-ae95-41dc-a561-7d7f10d9ac08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:40:32.697060 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953d47c7-ae95-41dc-a561-7d7f10d9ac08 admin admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:40:32.697343 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953d47c7-ae95-41dc-a561-7d7f10d9ac08 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:40:32.697620 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953d47c7-ae95-41dc-a561-7d7f10d9ac08 admin admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:40:32.698383 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-953d47c7-ae95-41dc-a561-7d7f10d9ac08 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:40:32.700032 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-953d47c7-ae95-41dc-a561-7d7f10d9ac08 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:40:32.726586 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 859/1728] 10.222.0.57 () {58 vars in 1121 bytes} [Fri Jul 1 16:40:32 2022] GET /identity/v3/roles => generated 1420 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 16:40:32.727903 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953d47c7-ae95-41dc-a561-7d7f10d9ac08 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:40:32.731942 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-953d47c7-ae95-41dc-a561-7d7f10d9ac08 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:40:32.761506 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953d47c7-ae95-41dc-a561-7d7f10d9ac08 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:40:32.779258 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b337971d-41ab-4d04-823c-c6a89e295537 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:40:32.780454 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b337971d-41ab-4d04-823c-c6a89e295537 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:40:32.780454 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b337971d-41ab-4d04-823c-c6a89e295537 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:40:32.780735 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b337971d-41ab-4d04-823c-c6a89e295537 admin admin] PATH_INFO: `/v3/projects/8e11ec92d7d24814be64e3dc5f1ae0a4` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:40:32.783156 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b337971d-41ab-4d04-823c-c6a89e295537 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8e11ec92d7d24814be64e3dc5f1ae0a4)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:40:32.785843 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b337971d-41ab-4d04-823c-c6a89e295537 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:40:32.845885 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4fc07db-74eb-4e07-92ac-f0b39775ee8f service neutron] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:40:32.869726 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14cd7eb-59a7-4ffd-8f47-5246c49e03dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:40:32.871213 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14cd7eb-59a7-4ffd-8f47-5246c49e03dc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:40:32.871213 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14cd7eb-59a7-4ffd-8f47-5246c49e03dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:40:32.871213 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14cd7eb-59a7-4ffd-8f47-5246c49e03dc admin admin] PATH_INFO: `/v3/users/10bbd9ddbcec4299b8635d6df0a41a4d` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:40:32.872587 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e14cd7eb-59a7-4ffd-8f47-5246c49e03dc admin admin] RBAC: Authorizing `identity:delete_user(user_id=10bbd9ddbcec4299b8635d6df0a41a4d)` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:40:32.874738 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e14cd7eb-59a7-4ffd-8f47-5246c49e03dc admin admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:40:32.911818 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-b337971d-41ab-4d04-823c-c6a89e295537 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8e11ec92d7d24814be64e3dc5f1ae0a4', 'request_id': 'req-b337971d-41ab-4d04-823c-c6a89e295537'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:40:32.923371 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-b337971d-41ab-4d04-823c-c6a89e295537 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8e11ec92d7d24814be64e3dc5f1ae0a4', 'request_id': 'req-b337971d-41ab-4d04-823c-c6a89e295537'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:40:32.943793 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-b337971d-41ab-4d04-823c-c6a89e295537 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8e11ec92d7d24814be64e3dc5f1ae0a4', 'request_id': 'req-b337971d-41ab-4d04-823c-c6a89e295537'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:40:32.980627 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-e14cd7eb-59a7-4ffd-8f47-5246c49e03dc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '10bbd9ddbcec4299b8635d6df0a41a4d'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:40:32.984735 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-b337971d-41ab-4d04-823c-c6a89e295537 admin admin] The token cache is being invalidate because project 8e11ec92d7d24814be64e3dc5f1ae0a4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80311) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 01 16:40:32.985214 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-b337971d-41ab-4d04-823c-c6a89e295537 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:40:32.985674 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:40:32.985913 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-b337971d-41ab-4d04-823c-c6a89e295537 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:40:32.993349 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 860/1729] 10.222.0.57 () {58 vars in 1232 bytes} [Fri Jul 1 16:40:32 2022] DELETE /identity/v3/projects/8e11ec92d7d24814be64e3dc5f1ae0a4 => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 16:40:32.994577 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b337971d-41ab-4d04-823c-c6a89e295537 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:40:33.007302 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-b337971d-41ab-4d04-823c-c6a89e295537 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:40:33.017423 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-e14cd7eb-59a7-4ffd-8f47-5246c49e03dc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '10bbd9ddbcec4299b8635d6df0a41a4d'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:40:33.039863 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-e14cd7eb-59a7-4ffd-8f47-5246c49e03dc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '10bbd9ddbcec4299b8635d6df0a41a4d'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:40:33.041920 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-e14cd7eb-59a7-4ffd-8f47-5246c49e03dc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '10bbd9ddbcec4299b8635d6df0a41a4d'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:40:33.044441 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-e14cd7eb-59a7-4ffd-8f47-5246c49e03dc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '10bbd9ddbcec4299b8635d6df0a41a4d'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:40:33.081838 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 870/1730] 10.222.0.57 () {58 vars in 1223 bytes} [Fri Jul 1 16:40:32 2022] DELETE /identity/v3/users/10bbd9ddbcec4299b8635d6df0a41a4d => generated 0 bytes in 471 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 16:40:33.082773 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14cd7eb-59a7-4ffd-8f47-5246c49e03dc admin admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:40:33.085998 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-e14cd7eb-59a7-4ffd-8f47-5246c49e03dc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:40:33.178373 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:40:33.261094 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14cd7eb-59a7-4ffd-8f47-5246c49e03dc admin admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:40:33.272693 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b337971d-41ab-4d04-823c-c6a89e295537 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:40:33.276404 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59fb5015-9099-4237-be1c-316a5cdaf477 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:40:33.277326 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fb5015-9099-4237-be1c-316a5cdaf477 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:40:33.277658 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fb5015-9099-4237-be1c-316a5cdaf477 admin admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:40:33.278054 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fb5015-9099-4237-be1c-316a5cdaf477 admin admin] PATH_INFO: `/v3/users/e5e4c87a0666464bbf3675103195e41d` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:40:33.281008 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59fb5015-9099-4237-be1c-316a5cdaf477 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e5e4c87a0666464bbf3675103195e41d)` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:40:33.283220 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59fb5015-9099-4237-be1c-316a5cdaf477 admin admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:40:33.294270 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df0d978-21a5-4cd5-bc46-207bb6dca7a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:40:33.296025 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df0d978-21a5-4cd5-bc46-207bb6dca7a8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:40:33.305626 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df0d978-21a5-4cd5-bc46-207bb6dca7a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:40:33.306306 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df0d978-21a5-4cd5-bc46-207bb6dca7a8 admin admin] PATH_INFO: `/v3/projects/9d29b085bfc54d3a94f6080bd0078690/users/8dbe1abec62c45fc941ffe30a4596a61/roles/983be40bdab04c478eaa90c6b6a46b14` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:40:33.375493 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df0d978-21a5-4cd5-bc46-207bb6dca7a8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9d29b085bfc54d3a94f6080bd0078690, user_id=8dbe1abec62c45fc941ffe30a4596a61, role_id=983be40bdab04c478eaa90c6b6a46b14)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:40:33.399030 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-59fb5015-9099-4237-be1c-316a5cdaf477 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e5e4c87a0666464bbf3675103195e41d'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:40:33.400895 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df0d978-21a5-4cd5-bc46-207bb6dca7a8 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:40:33.423764 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-59fb5015-9099-4237-be1c-316a5cdaf477 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e5e4c87a0666464bbf3675103195e41d'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:40:33.443282 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-59fb5015-9099-4237-be1c-316a5cdaf477 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e5e4c87a0666464bbf3675103195e41d'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:40:33.445657 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-59fb5015-9099-4237-be1c-316a5cdaf477 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e5e4c87a0666464bbf3675103195e41d'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:40:33.446537 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-59fb5015-9099-4237-be1c-316a5cdaf477 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e5e4c87a0666464bbf3675103195e41d'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:40:33.448068 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 861/1731] 10.222.0.57 () {60 vars in 1482 bytes} [Fri Jul 1 16:40:32 2022] PUT /identity/v3/projects/9d29b085bfc54d3a94f6080bd0078690/users/8dbe1abec62c45fc941ffe30a4596a61/roles/983be40bdab04c478eaa90c6b6a46b14 => generated 0 bytes in 454 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 16:40:33.459785 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e830de-0c8f-44cf-a9a6-aac7336db3f7 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:40:33.460176 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e830de-0c8f-44cf-a9a6-aac7336db3f7 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:40:33.460556 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e830de-0c8f-44cf-a9a6-aac7336db3f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:40:33.486738 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 871/1732] 10.222.0.57 () {58 vars in 1223 bytes} [Fri Jul 1 16:40:33 2022] DELETE /identity/v3/users/e5e4c87a0666464bbf3675103195e41d => generated 0 bytes in 404 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 16:40:33.587769 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-e1e830de-0c8f-44cf-a9a6-aac7336db3f7 None None] MFA Rules not processed for user `8dbe1abec62c45fc941ffe30a4596a61`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:40:33.594730 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-e1e830de-0c8f-44cf-a9a6-aac7336db3f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:40:33.841963 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 862/1733] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 1 16:40:33 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 384 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 16:40:33.846631 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59fb5015-9099-4237-be1c-316a5cdaf477 admin admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:40:33.848949 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-59fb5015-9099-4237-be1c-316a5cdaf477 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:40:34.015072 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:40:34.061092 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59fb5015-9099-4237-be1c-316a5cdaf477 admin admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:40:34.079087 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ee809a-346b-4683-8031-786dc37abce0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:40:34.079643 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ee809a-346b-4683-8031-786dc37abce0 admin admin] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:40:34.079929 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ee809a-346b-4683-8031-786dc37abce0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:40:34.080210 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ee809a-346b-4683-8031-786dc37abce0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:40:34.080987 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ee809a-346b-4683-8031-786dc37abce0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:40:34.083062 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ee809a-346b-4683-8031-786dc37abce0 admin admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:40:34.146042 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 872/1734] 10.222.0.57 () {60 vars in 1152 bytes} [Fri Jul 1 16:40:33 2022] POST /identity/v3/projects => generated 373 bytes in 300 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 16:40:34.149810 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e830de-0c8f-44cf-a9a6-aac7336db3f7 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:40:34.167320 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e830de-0c8f-44cf-a9a6-aac7336db3f7 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:40:34.314121 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06064367-1907-4ca2-a19a-7ab8e94e057a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:40:34.314965 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06064367-1907-4ca2-a19a-7ab8e94e057a admin admin] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:40:34.315378 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06064367-1907-4ca2-a19a-7ab8e94e057a admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:40:34.315674 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06064367-1907-4ca2-a19a-7ab8e94e057a admin admin] PATH_INFO: `/v3/users` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:40:34.316551 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06064367-1907-4ca2-a19a-7ab8e94e057a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:40:34.318883 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06064367-1907-4ca2-a19a-7ab8e94e057a admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:40:34.356792 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 863/1735] 10.222.0.57 () {60 vars in 1143 bytes} [Fri Jul 1 16:40:34 2022] POST /identity/v3/users => generated 293 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 16:40:34.360584 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ee809a-346b-4683-8031-786dc37abce0 admin admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:40:34.376504 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ee809a-346b-4683-8031-786dc37abce0 admin admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:40:34.380943 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06064367-1907-4ca2-a19a-7ab8e94e057a admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:40:34.383393 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-06064367-1907-4ca2-a19a-7ab8e94e057a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:40:34.398400 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1e08d5-d887-4afd-a8e4-183361ac261d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:40:34.399083 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1e08d5-d887-4afd-a8e4-183361ac261d admin admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:40:34.399366 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1e08d5-d887-4afd-a8e4-183361ac261d admin admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:40:34.399656 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1e08d5-d887-4afd-a8e4-183361ac261d admin admin] PATH_INFO: `/v3/roles` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:40:34.400398 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c1e08d5-d887-4afd-a8e4-183361ac261d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:40:34.402148 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c1e08d5-d887-4afd-a8e4-183361ac261d admin admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:40:34.441348 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 873/1736] 10.222.0.57 () {58 vars in 1121 bytes} [Fri Jul 1 16:40:34 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 16:40:34.442550 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1e08d5-d887-4afd-a8e4-183361ac261d admin admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:40:34.464896 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1e08d5-d887-4afd-a8e4-183361ac261d admin admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:40:34.483786 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34aea903-032a-46af-ba7b-414bb0737407 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:40:34.485230 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34aea903-032a-46af-ba7b-414bb0737407 admin admin] REQUEST_METHOD: `PUT` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:40:34.485230 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34aea903-032a-46af-ba7b-414bb0737407 admin admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:40:34.485526 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34aea903-032a-46af-ba7b-414bb0737407 admin admin] PATH_INFO: `/v3/projects/01c8d496300041168b3f96a865bcef1c/users/1409afa52a224eb18a51fdba56a04dff/roles/84ee739146934c1ab32c501ff8093644` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:40:34.516313 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:40:34.519064 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34aea903-032a-46af-ba7b-414bb0737407 admin admin] RBAC: Authorizing `identity:create_grant(project_id=01c8d496300041168b3f96a865bcef1c, user_id=1409afa52a224eb18a51fdba56a04dff, role_id=84ee739146934c1ab32c501ff8093644)` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:40:34.526112 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34aea903-032a-46af-ba7b-414bb0737407 admin admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:40:34.564270 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 874/1737] 10.222.0.57 () {60 vars in 1482 bytes} [Fri Jul 1 16:40:34 2022] PUT /identity/v3/projects/01c8d496300041168b3f96a865bcef1c/users/1409afa52a224eb18a51fdba56a04dff/roles/84ee739146934c1ab32c501ff8093644 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 16:40:34.565922 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34aea903-032a-46af-ba7b-414bb0737407 admin admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:40:34.576312 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-34aea903-032a-46af-ba7b-414bb0737407 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:40:34.591969 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06064367-1907-4ca2-a19a-7ab8e94e057a admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:40:34.706160 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fcabc1d-d10e-48d2-8ee6-5855ccb5d2bc service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc38af18dec4e9b8143df23af6aad43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0788e6725a84e57a4174284c1845755', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:40:34.707025 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcabc1d-d10e-48d2-8ee6-5855ccb5d2bc service placement] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:40:34.707207 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcabc1d-d10e-48d2-8ee6-5855ccb5d2bc service placement] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:40:34.709657 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcabc1d-d10e-48d2-8ee6-5855ccb5d2bc service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:40:34.713662 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-7fcabc1d-d10e-48d2-8ee6-5855ccb5d2bc service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:40:34.728227 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fcabc1d-d10e-48d2-8ee6-5855ccb5d2bc service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:40:34.729874 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fcabc1d-d10e-48d2-8ee6-5855ccb5d2bc service placement] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:40:34.733960 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34aea903-032a-46af-ba7b-414bb0737407 admin admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:40:34.747990 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 864/1738] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 1 16:40:34 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 16:40:34.748927 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fcabc1d-d10e-48d2-8ee6-5855ccb5d2bc service placement] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:40:34.750471 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3164f413-1d49-4e43-a2fd-d786bb9f1b99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:40:34.753914 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3164f413-1d49-4e43-a2fd-d786bb9f1b99 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:40:34.754379 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3164f413-1d49-4e43-a2fd-d786bb9f1b99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:40:34.757302 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3164f413-1d49-4e43-a2fd-d786bb9f1b99 admin admin] PATH_INFO: `/v3/projects/53e556371eec40d398ae089d2714a856` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:40:34.761349 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3164f413-1d49-4e43-a2fd-d786bb9f1b99 admin admin] RBAC: Authorizing `identity:delete_project(project_id=53e556371eec40d398ae089d2714a856)` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:40:34.763984 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3164f413-1d49-4e43-a2fd-d786bb9f1b99 admin admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:40:34.778566 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fcabc1d-d10e-48d2-8ee6-5855ccb5d2bc service placement] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:40:34.819205 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be95adb6-4af0-4f14-b7c3-07a8670f1fc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:40:34.819916 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be95adb6-4af0-4f14-b7c3-07a8670f1fc4 admin admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:40:34.820206 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be95adb6-4af0-4f14-b7c3-07a8670f1fc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:40:34.820496 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be95adb6-4af0-4f14-b7c3-07a8670f1fc4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:40:34.821248 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be95adb6-4af0-4f14-b7c3-07a8670f1fc4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:40:34.826054 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be95adb6-4af0-4f14-b7c3-07a8670f1fc4 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:40:34.848970 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 865/1739] 10.222.0.57 () {58 vars in 1121 bytes} [Fri Jul 1 16:40:34 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 16:40:34.856274 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be95adb6-4af0-4f14-b7c3-07a8670f1fc4 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:40:34.907766 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be95adb6-4af0-4f14-b7c3-07a8670f1fc4 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:40:34.933325 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-3164f413-1d49-4e43-a2fd-d786bb9f1b99 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '53e556371eec40d398ae089d2714a856', 'request_id': 'req-3164f413-1d49-4e43-a2fd-d786bb9f1b99'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:40:34.951803 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-3164f413-1d49-4e43-a2fd-d786bb9f1b99 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '53e556371eec40d398ae089d2714a856', 'request_id': 'req-3164f413-1d49-4e43-a2fd-d786bb9f1b99'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:40:34.971287 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-3164f413-1d49-4e43-a2fd-d786bb9f1b99 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '53e556371eec40d398ae089d2714a856', 'request_id': 'req-3164f413-1d49-4e43-a2fd-d786bb9f1b99'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:40:35.036264 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-3164f413-1d49-4e43-a2fd-d786bb9f1b99 admin admin] The token cache is being invalidate because project 53e556371eec40d398ae089d2714a856 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80312) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 01 16:40:35.036561 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-3164f413-1d49-4e43-a2fd-d786bb9f1b99 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:40:35.039129 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-3164f413-1d49-4e43-a2fd-d786bb9f1b99 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:40:35.045686 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 875/1740] 10.222.0.57 () {58 vars in 1232 bytes} [Fri Jul 1 16:40:34 2022] DELETE /identity/v3/projects/53e556371eec40d398ae089d2714a856 => generated 0 bytes in 481 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 16:40:35.051694 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d434f2a-7413-408d-a3e7-b6565912e417 None None] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:40:35.061826 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d434f2a-7413-408d-a3e7-b6565912e417 None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:40:35.062264 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d434f2a-7413-408d-a3e7-b6565912e417 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:40:35.110700 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.auth.core [None req-9d434f2a-7413-408d-a3e7-b6565912e417 None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80312) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:40:35.113372 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-9d434f2a-7413-408d-a3e7-b6565912e417 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:40:35.196988 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:40:35.285180 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098f4d35-34cf-4d69-86d1-938b4aacc562 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:40:35.291092 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098f4d35-34cf-4d69-86d1-938b4aacc562 admin admin] REQUEST_METHOD: `PUT` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:40:35.291567 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098f4d35-34cf-4d69-86d1-938b4aacc562 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:40:35.292065 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098f4d35-34cf-4d69-86d1-938b4aacc562 admin admin] PATH_INFO: `/v3/projects/01c8d496300041168b3f96a865bcef1c/users/1409afa52a224eb18a51fdba56a04dff/roles/84ee739146934c1ab32c501ff8093644` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:40:35.307326 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-098f4d35-34cf-4d69-86d1-938b4aacc562 admin admin] RBAC: Authorizing `identity:create_grant(project_id=01c8d496300041168b3f96a865bcef1c, user_id=1409afa52a224eb18a51fdba56a04dff, role_id=84ee739146934c1ab32c501ff8093644)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:40:35.309865 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-098f4d35-34cf-4d69-86d1-938b4aacc562 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:40:35.363590 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 866/1741] 10.222.0.57 () {60 vars in 1482 bytes} [Fri Jul 1 16:40:34 2022] PUT /identity/v3/projects/01c8d496300041168b3f96a865bcef1c/users/1409afa52a224eb18a51fdba56a04dff/roles/84ee739146934c1ab32c501ff8093644 => generated 0 bytes in 507 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 16:40:35.376435 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 876/1742] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 1 16:40:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 326 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 16:40:35.383813 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25884f9a-613d-44aa-9368-ec1ee5bd7951 None None] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:40:35.389681 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25884f9a-613d-44aa-9368-ec1ee5bd7951 None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:40:35.389681 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25884f9a-613d-44aa-9368-ec1ee5bd7951 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:40:35.409346 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6cb3a8-aa7c-49e7-b451-a7e1bb1c8076 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:40:35.409598 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6cb3a8-aa7c-49e7-b451-a7e1bb1c8076 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:40:35.409746 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6cb3a8-aa7c-49e7-b451-a7e1bb1c8076 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:40:35.423115 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.auth.core [None req-25884f9a-613d-44aa-9368-ec1ee5bd7951 None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80312) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:40:35.424319 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-25884f9a-613d-44aa-9368-ec1ee5bd7951 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:40:35.538432 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-1d6cb3a8-aa7c-49e7-b451-a7e1bb1c8076 None None] MFA Rules not processed for user `1409afa52a224eb18a51fdba56a04dff`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:40:35.539692 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-1d6cb3a8-aa7c-49e7-b451-a7e1bb1c8076 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:40:35.625379 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 877/1743] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 1 16:40:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 242 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 16:40:35.630192 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25884f9a-613d-44aa-9368-ec1ee5bd7951 None None] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:40:35.654071 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25884f9a-613d-44aa-9368-ec1ee5bd7951 None None] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:40:35.673196 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcdbc1e6-7630-4409-aa1c-4e82849d94a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:40:35.673892 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdbc1e6-7630-4409-aa1c-4e82849d94a0 admin admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:40:35.675572 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdbc1e6-7630-4409-aa1c-4e82849d94a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:40:35.675892 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdbc1e6-7630-4409-aa1c-4e82849d94a0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:40:35.676660 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcdbc1e6-7630-4409-aa1c-4e82849d94a0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80312) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:40:35.676979 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcdbc1e6-7630-4409-aa1c-4e82849d94a0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:40:35.679101 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcdbc1e6-7630-4409-aa1c-4e82849d94a0 admin admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:40:35.717640 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 878/1744] 10.222.0.57 () {58 vars in 1152 bytes} [Fri Jul 1 16:40:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:40:35.718626 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcdbc1e6-7630-4409-aa1c-4e82849d94a0 admin admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:40:35.738926 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:40:35.748494 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcdbc1e6-7630-4409-aa1c-4e82849d94a0 admin admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:40:35.779926 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82e6cca-01ec-4fba-b081-a6d79a010c49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:40:35.780871 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82e6cca-01ec-4fba-b081-a6d79a010c49 admin admin] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:40:35.781285 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82e6cca-01ec-4fba-b081-a6d79a010c49 admin admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:40:35.781572 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82e6cca-01ec-4fba-b081-a6d79a010c49 admin admin] PATH_INFO: `/v3/projects` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:40:35.782367 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d82e6cca-01ec-4fba-b081-a6d79a010c49 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:40:35.787175 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d82e6cca-01ec-4fba-b081-a6d79a010c49 admin admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:40:35.842185 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 867/1745] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 1 16:40:35 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 461 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 16:40:35.857242 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6cb3a8-aa7c-49e7-b451-a7e1bb1c8076 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:40:35.858885 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-1d6cb3a8-aa7c-49e7-b451-a7e1bb1c8076 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:40:35.916424 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 879/1746] 10.222.0.57 () {60 vars in 1152 bytes} [Fri Jul 1 16:40:35 2022] POST /identity/v3/projects => generated 381 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 16:40:35.918917 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82e6cca-01ec-4fba-b081-a6d79a010c49 admin admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:40:36.029708 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6cb3a8-aa7c-49e7-b451-a7e1bb1c8076 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:40:36.089967 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82e6cca-01ec-4fba-b081-a6d79a010c49 admin admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:40:36.096111 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b65bb67-b09a-4715-b2a2-0090e6d98350 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:40:36.097297 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b65bb67-b09a-4715-b2a2-0090e6d98350 admin admin] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:40:36.098042 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b65bb67-b09a-4715-b2a2-0090e6d98350 admin admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:40:36.098315 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b65bb67-b09a-4715-b2a2-0090e6d98350 admin admin] PATH_INFO: `/v3/users` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:40:36.099246 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b65bb67-b09a-4715-b2a2-0090e6d98350 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:40:36.101443 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b65bb67-b09a-4715-b2a2-0090e6d98350 admin admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:40:36.135985 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2aee18-32d1-4269-a761-eff23d7bb17e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbddf971a2354bbdbb9da1d63f3abf88', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0788e6725a84e57a4174284c1845755', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:40:36.136680 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2aee18-32d1-4269-a761-eff23d7bb17e service cinder] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:40:36.136970 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2aee18-32d1-4269-a761-eff23d7bb17e service cinder] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:40:36.137411 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2aee18-32d1-4269-a761-eff23d7bb17e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:40:36.142645 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 880/1747] 10.222.0.57 () {60 vars in 1143 bytes} [Fri Jul 1 16:40:35 2022] POST /identity/v3/users => generated 291 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 16:40:36.144760 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82931df-9ef9-454a-8041-510fa1bb37fc None None] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:40:36.145055 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82931df-9ef9-454a-8041-510fa1bb37fc None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:40:36.145572 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82931df-9ef9-454a-8041-510fa1bb37fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:40:36.223762 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.auth.core [None req-b82931df-9ef9-454a-8041-510fa1bb37fc None None] MFA Rules not processed for user `e471b31763074167ba81c2c25f39b857`. Rule list: `[]` (Enabled: `True`). {{(pid=80312) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:40:36.225026 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-b82931df-9ef9-454a-8041-510fa1bb37fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:40:36.251846 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:40:36.320877 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa2aee18-32d1-4269-a761-eff23d7bb17e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:40:36.325164 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa2aee18-32d1-4269-a761-eff23d7bb17e service cinder] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:40:36.355523 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 868/1748] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 1 16:40:35 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 499 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 16:40:36.356505 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2aee18-32d1-4269-a761-eff23d7bb17e service cinder] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:40:36.383946 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 881/1749] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 1 16:40:36 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 241 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 16:40:36.384615 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82931df-9ef9-454a-8041-510fa1bb37fc None None] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:40:36.386074 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-b82931df-9ef9-454a-8041-510fa1bb37fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:40:36.389614 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2aee18-32d1-4269-a761-eff23d7bb17e service cinder] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:40:36.416614 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162e2bf4-e681-4af4-8dd5-7f894f895843 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:40:36.417445 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162e2bf4-e681-4af4-8dd5-7f894f895843 admin admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:40:36.417852 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162e2bf4-e681-4af4-8dd5-7f894f895843 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:40:36.419000 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162e2bf4-e681-4af4-8dd5-7f894f895843 admin admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:40:36.420159 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-162e2bf4-e681-4af4-8dd5-7f894f895843 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:40:36.422036 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-162e2bf4-e681-4af4-8dd5-7f894f895843 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:40:36.441906 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82931df-9ef9-454a-8041-510fa1bb37fc None None] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:40:36.447491 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 869/1750] 10.222.0.57 () {58 vars in 1121 bytes} [Fri Jul 1 16:40:36 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 16:40:36.448385 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162e2bf4-e681-4af4-8dd5-7f894f895843 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:40:36.450332 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-162e2bf4-e681-4af4-8dd5-7f894f895843 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:40:36.457893 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15368443-6864-431e-85bd-846ba7d5d11c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:40:36.460873 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15368443-6864-431e-85bd-846ba7d5d11c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:40:36.461313 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15368443-6864-431e-85bd-846ba7d5d11c admin admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:40:36.463877 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15368443-6864-431e-85bd-846ba7d5d11c admin admin] PATH_INFO: `/v3/projects/cfd794279b654d3c990f6d84f4dad747` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:40:36.467459 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15368443-6864-431e-85bd-846ba7d5d11c admin admin] RBAC: Authorizing `identity:delete_project(project_id=cfd794279b654d3c990f6d84f4dad747)` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:40:36.478990 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15368443-6864-431e-85bd-846ba7d5d11c admin admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:40:36.547311 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:40:36.605495 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-15368443-6864-431e-85bd-846ba7d5d11c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cfd794279b654d3c990f6d84f4dad747', 'request_id': 'req-15368443-6864-431e-85bd-846ba7d5d11c'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:40:36.628916 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-15368443-6864-431e-85bd-846ba7d5d11c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cfd794279b654d3c990f6d84f4dad747', 'request_id': 'req-15368443-6864-431e-85bd-846ba7d5d11c'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:40:36.637958 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-15368443-6864-431e-85bd-846ba7d5d11c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cfd794279b654d3c990f6d84f4dad747', 'request_id': 'req-15368443-6864-431e-85bd-846ba7d5d11c'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:40:36.645973 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162e2bf4-e681-4af4-8dd5-7f894f895843 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:40:36.679217 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-15368443-6864-431e-85bd-846ba7d5d11c admin admin] The token cache is being invalidate because project cfd794279b654d3c990f6d84f4dad747 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80312) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 01 16:40:36.680005 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-15368443-6864-431e-85bd-846ba7d5d11c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:40:36.680396 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-15368443-6864-431e-85bd-846ba7d5d11c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:40:36.683786 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 882/1751] 10.222.0.57 () {58 vars in 1232 bytes} [Fri Jul 1 16:40:36 2022] DELETE /identity/v3/projects/cfd794279b654d3c990f6d84f4dad747 => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 16:40:36.685008 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15368443-6864-431e-85bd-846ba7d5d11c admin admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:40:36.691971 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-15368443-6864-431e-85bd-846ba7d5d11c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:40:36.818756 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2960a347-ba49-45fb-836f-3f0ac2dd3a88 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a50f745b31941b7a7265c0fc852f48e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0788e6725a84e57a4174284c1845755', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:40:36.819557 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2960a347-ba49-45fb-836f-3f0ac2dd3a88 service nova] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:40:36.819919 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2960a347-ba49-45fb-836f-3f0ac2dd3a88 service nova] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:40:36.820276 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2960a347-ba49-45fb-836f-3f0ac2dd3a88 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:40:36.825615 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-2960a347-ba49-45fb-836f-3f0ac2dd3a88 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:40:36.901001 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15368443-6864-431e-85bd-846ba7d5d11c admin admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:40:36.914075 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153c364c-24fc-4da4-8cf3-aa1c37d1b6cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:40:36.915002 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153c364c-24fc-4da4-8cf3-aa1c37d1b6cb admin admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:40:36.915356 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153c364c-24fc-4da4-8cf3-aa1c37d1b6cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:40:36.915755 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153c364c-24fc-4da4-8cf3-aa1c37d1b6cb admin admin] PATH_INFO: `/v3/roles` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:40:36.916773 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-153c364c-24fc-4da4-8cf3-aa1c37d1b6cb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:40:36.919581 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-153c364c-24fc-4da4-8cf3-aa1c37d1b6cb admin admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:40:36.935577 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2960a347-ba49-45fb-836f-3f0ac2dd3a88 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:40:36.942712 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2960a347-ba49-45fb-836f-3f0ac2dd3a88 service nova] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:40:36.952451 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 883/1752] 10.222.0.57 () {58 vars in 1121 bytes} [Fri Jul 1 16:40:36 2022] GET /identity/v3/roles => generated 1420 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 16:40:36.953645 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153c364c-24fc-4da4-8cf3-aa1c37d1b6cb admin admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:40:36.954018 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 870/1753] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 1 16:40:36 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 506 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 16:40:36.955738 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ec18d6-7371-41f6-aeda-3240f70ad2c2 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:40:36.956878 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ec18d6-7371-41f6-aeda-3240f70ad2c2 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:40:36.957414 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ec18d6-7371-41f6-aeda-3240f70ad2c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:40:36.977364 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-153c364c-24fc-4da4-8cf3-aa1c37d1b6cb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:40:36.991637 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-f7ec18d6-7371-41f6-aeda-3240f70ad2c2 None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:40:36.994691 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-f7ec18d6-7371-41f6-aeda-3240f70ad2c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:40:37.043955 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 871/1754] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 1 16:40:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 16:40:37.047036 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ec18d6-7371-41f6-aeda-3240f70ad2c2 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:40:37.076830 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ec18d6-7371-41f6-aeda-3240f70ad2c2 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:40:37.094707 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66991227-b474-4ab4-9902-c2199c070ad3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:40:37.095817 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66991227-b474-4ab4-9902-c2199c070ad3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:40:37.096170 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66991227-b474-4ab4-9902-c2199c070ad3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:40:37.096571 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66991227-b474-4ab4-9902-c2199c070ad3 admin admin] PATH_INFO: `/v3/projects/943fef40af6f42a3bdde3775c2ca0297/users/be0bed3de52945dab254529167d4c492/roles/983be40bdab04c478eaa90c6b6a46b14` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:40:37.124615 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:40:37.155323 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66991227-b474-4ab4-9902-c2199c070ad3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=943fef40af6f42a3bdde3775c2ca0297, user_id=be0bed3de52945dab254529167d4c492, role_id=983be40bdab04c478eaa90c6b6a46b14)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:40:37.166049 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66991227-b474-4ab4-9902-c2199c070ad3 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:40:37.206470 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 872/1755] 10.222.0.57 () {60 vars in 1482 bytes} [Fri Jul 1 16:40:37 2022] PUT /identity/v3/projects/943fef40af6f42a3bdde3775c2ca0297/users/be0bed3de52945dab254529167d4c492/roles/983be40bdab04c478eaa90c6b6a46b14 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 16:40:37.206470 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66991227-b474-4ab4-9902-c2199c070ad3 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:40:37.206470 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-66991227-b474-4ab4-9902-c2199c070ad3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:40:37.207058 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153c364c-24fc-4da4-8cf3-aa1c37d1b6cb admin admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:40:37.322197 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:40:37.347376 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f8d41b-adbc-4d01-89ec-645453d91bbb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbddf971a2354bbdbb9da1d63f3abf88', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0788e6725a84e57a4174284c1845755', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:40:37.349935 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f8d41b-adbc-4d01-89ec-645453d91bbb service cinder] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:40:37.350480 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f8d41b-adbc-4d01-89ec-645453d91bbb service cinder] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:40:37.350918 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f8d41b-adbc-4d01-89ec-645453d91bbb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:40:37.356952 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-18f8d41b-adbc-4d01-89ec-645453d91bbb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:40:37.379382 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66991227-b474-4ab4-9902-c2199c070ad3 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:40:37.400296 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63147f1-e274-4509-baa6-cffe89deb301 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:40:37.401401 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63147f1-e274-4509-baa6-cffe89deb301 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:40:37.403631 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63147f1-e274-4509-baa6-cffe89deb301 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:40:37.404030 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63147f1-e274-4509-baa6-cffe89deb301 admin admin] PATH_INFO: `/v3/users/923dd887fef04df1baa5377db1f6c5e1` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:40:37.408480 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63147f1-e274-4509-baa6-cffe89deb301 admin admin] RBAC: Authorizing `identity:delete_user(user_id=923dd887fef04df1baa5377db1f6c5e1)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:40:37.415116 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63147f1-e274-4509-baa6-cffe89deb301 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:40:37.516354 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f8d41b-adbc-4d01-89ec-645453d91bbb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:40:37.520748 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f8d41b-adbc-4d01-89ec-645453d91bbb service cinder] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:40:37.560865 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 884/1756] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 1 16:40:36 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 607 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 16:40:37.562666 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a54369f-04cc-4fce-bfa2-78f4f0a0c55c None None] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:40:37.569660 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-b63147f1-e274-4509-baa6-cffe89deb301 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '923dd887fef04df1baa5377db1f6c5e1'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:40:37.570767 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-b63147f1-e274-4509-baa6-cffe89deb301 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '923dd887fef04df1baa5377db1f6c5e1'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:40:37.571380 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a54369f-04cc-4fce-bfa2-78f4f0a0c55c None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:40:37.571683 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a54369f-04cc-4fce-bfa2-78f4f0a0c55c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:40:37.571924 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-b63147f1-e274-4509-baa6-cffe89deb301 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '923dd887fef04df1baa5377db1f6c5e1'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:40:37.588703 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-b63147f1-e274-4509-baa6-cffe89deb301 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '923dd887fef04df1baa5377db1f6c5e1'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:40:37.606788 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-b63147f1-e274-4509-baa6-cffe89deb301 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '923dd887fef04df1baa5377db1f6c5e1'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:40:37.629013 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.auth.core [None req-9a54369f-04cc-4fce-bfa2-78f4f0a0c55c None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80312) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:40:37.630275 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-9a54369f-04cc-4fce-bfa2-78f4f0a0c55c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:40:37.658436 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 885/1757] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 1 16:40:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 16:40:37.659911 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385b5729-5c30-4ad0-9933-6557a1e866ad None None] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:40:37.660205 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385b5729-5c30-4ad0-9933-6557a1e866ad None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:40:37.660487 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385b5729-5c30-4ad0-9933-6557a1e866ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:40:37.662578 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 873/1758] 10.222.0.57 () {58 vars in 1223 bytes} [Fri Jul 1 16:40:37 2022] DELETE /identity/v3/users/923dd887fef04df1baa5377db1f6c5e1 => generated 0 bytes in 463 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 16:40:37.665340 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63147f1-e274-4509-baa6-cffe89deb301 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:40:37.720287 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.auth.core [None req-385b5729-5c30-4ad0-9933-6557a1e866ad None None] MFA Rules not processed for user `be0bed3de52945dab254529167d4c492`. Rule list: `[]` (Enabled: `True`). {{(pid=80312) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:40:37.721859 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-385b5729-5c30-4ad0-9933-6557a1e866ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:40:37.828871 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 886/1759] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 1 16:40:37 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 16:40:37.830252 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e1a75c-1cff-44f7-b1d5-df80bc1490b5 None None] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:40:37.830438 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e1a75c-1cff-44f7-b1d5-df80bc1490b5 None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:40:37.830782 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e1a75c-1cff-44f7-b1d5-df80bc1490b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:40:37.872886 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63147f1-e274-4509-baa6-cffe89deb301 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:40:37.876181 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.auth.core [None req-71e1a75c-1cff-44f7-b1d5-df80bc1490b5 None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80312) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:40:37.877415 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-71e1a75c-1cff-44f7-b1d5-df80bc1490b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:40:37.888531 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a899bea9-fc4e-49b7-a74b-5d3c7c35338e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:40:37.889225 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a899bea9-fc4e-49b7-a74b-5d3c7c35338e admin admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:40:37.889515 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a899bea9-fc4e-49b7-a74b-5d3c7c35338e admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:40:37.889812 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a899bea9-fc4e-49b7-a74b-5d3c7c35338e admin admin] PATH_INFO: `/v3/domains` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:40:37.890579 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a899bea9-fc4e-49b7-a74b-5d3c7c35338e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:40:37.891489 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a899bea9-fc4e-49b7-a74b-5d3c7c35338e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:40:37.893150 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a899bea9-fc4e-49b7-a74b-5d3c7c35338e admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:40:37.901188 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 887/1760] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 1 16:40:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 72 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 16:40:37.902046 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e1a75c-1cff-44f7-b1d5-df80bc1490b5 None None] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:40:37.908079 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-71e1a75c-1cff-44f7-b1d5-df80bc1490b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:40:37.924968 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 874/1761] 10.222.0.57 () {58 vars in 1152 bytes} [Fri Jul 1 16:40:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:40:37.925858 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a899bea9-fc4e-49b7-a74b-5d3c7c35338e admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:40:37.930039 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-a899bea9-fc4e-49b7-a74b-5d3c7c35338e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:40:37.937162 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e1a75c-1cff-44f7-b1d5-df80bc1490b5 None None] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:40:37.950549 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6431e002-d1fc-4a77-af0b-dcd5f534678a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:40:37.951237 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6431e002-d1fc-4a77-af0b-dcd5f534678a admin admin] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:40:37.951526 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6431e002-d1fc-4a77-af0b-dcd5f534678a admin admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:40:37.952252 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6431e002-d1fc-4a77-af0b-dcd5f534678a admin admin] PATH_INFO: `/v3/projects` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:40:37.952516 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6431e002-d1fc-4a77-af0b-dcd5f534678a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:40:37.957665 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6431e002-d1fc-4a77-af0b-dcd5f534678a admin admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:40:37.994769 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:40:38.029505 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 888/1762] 10.222.0.57 () {60 vars in 1152 bytes} [Fri Jul 1 16:40:37 2022] POST /identity/v3/projects => generated 379 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 16:40:38.032202 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6431e002-d1fc-4a77-af0b-dcd5f534678a admin admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:40:38.085705 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a899bea9-fc4e-49b7-a74b-5d3c7c35338e admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:40:38.099425 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:40:38.195603 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6431e002-d1fc-4a77-af0b-dcd5f534678a admin admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:40:38.207084 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab312aa-a29f-459d-82e4-425708f3e46f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:40:38.207774 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab312aa-a29f-459d-82e4-425708f3e46f admin admin] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:40:38.208063 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab312aa-a29f-459d-82e4-425708f3e46f admin admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:40:38.208339 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab312aa-a29f-459d-82e4-425708f3e46f admin admin] PATH_INFO: `/v3/projects` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:40:38.209305 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab312aa-a29f-459d-82e4-425708f3e46f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:40:38.213274 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab312aa-a29f-459d-82e4-425708f3e46f admin admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:40:38.215213 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e110a418-e661-4210-97b8-36cc68752eb9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ed200ce29684c5f80354fa9a8638fe2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0788e6725a84e57a4174284c1845755', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:40:38.215929 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e110a418-e661-4210-97b8-36cc68752eb9 service neutron] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:40:38.216169 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e110a418-e661-4210-97b8-36cc68752eb9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:40:38.216449 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e110a418-e661-4210-97b8-36cc68752eb9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:40:38.234207 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e110a418-e661-4210-97b8-36cc68752eb9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:40:38.237999 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e110a418-e661-4210-97b8-36cc68752eb9 service neutron] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:40:38.279932 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 875/1763] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 1 16:40:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 16:40:38.282701 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e110a418-e661-4210-97b8-36cc68752eb9 service neutron] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:40:38.290551 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 889/1764] 10.222.0.57 () {60 vars in 1152 bytes} [Fri Jul 1 16:40:38 2022] POST /identity/v3/projects => generated 365 bytes in 259 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 16:40:38.292844 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5a1af2-51df-4bfb-bff2-acc8f3d7067a None None] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:40:38.293155 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5a1af2-51df-4bfb-bff2-acc8f3d7067a None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:40:38.293442 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5a1af2-51df-4bfb-bff2-acc8f3d7067a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:40:38.317547 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e110a418-e661-4210-97b8-36cc68752eb9 service neutron] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:40:38.373359 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.auth.core [None req-9b5a1af2-51df-4bfb-bff2-acc8f3d7067a None None] MFA Rules not processed for user `ea27a708e2c84ccc905ae783524bd12f`. Rule list: `[]` (Enabled: `True`). {{(pid=80312) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:40:38.375790 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-9b5a1af2-51df-4bfb-bff2-acc8f3d7067a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:40:38.516547 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68701f3d-00ed-4570-be88-3bcab21c2a8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:40:38.517247 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68701f3d-00ed-4570-be88-3bcab21c2a8c admin admin] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:40:38.517519 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68701f3d-00ed-4570-be88-3bcab21c2a8c admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:40:38.517795 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68701f3d-00ed-4570-be88-3bcab21c2a8c admin admin] PATH_INFO: `/v3/users` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:40:38.518640 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68701f3d-00ed-4570-be88-3bcab21c2a8c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:40:38.520588 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68701f3d-00ed-4570-be88-3bcab21c2a8c admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:40:38.531232 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 890/1765] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 1 16:40:38 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 240 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 16:40:38.532270 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5a1af2-51df-4bfb-bff2-acc8f3d7067a None None] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:40:38.564473 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5a1af2-51df-4bfb-bff2-acc8f3d7067a None None] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:40:38.583825 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3609ad2-65f1-4da7-9613-842d4ceb6225 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:40:38.584778 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3609ad2-65f1-4da7-9613-842d4ceb6225 admin admin] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:40:38.585164 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3609ad2-65f1-4da7-9613-842d4ceb6225 admin admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:40:38.585365 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3609ad2-65f1-4da7-9613-842d4ceb6225 admin admin] PATH_INFO: `/v3/users` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:40:38.586071 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3609ad2-65f1-4da7-9613-842d4ceb6225 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:40:38.588855 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3609ad2-65f1-4da7-9613-842d4ceb6225 admin admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:40:38.595909 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 876/1766] 10.222.0.57 () {60 vars in 1143 bytes} [Fri Jul 1 16:40:38 2022] POST /identity/v3/users => generated 296 bytes in 310 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 16:40:38.595909 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6985d35-5dac-401f-814d-74006728376e None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:40:38.595909 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6985d35-5dac-401f-814d-74006728376e None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:40:38.595909 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6985d35-5dac-401f-814d-74006728376e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:40:38.644695 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 891/1767] 10.222.0.57 () {60 vars in 1143 bytes} [Fri Jul 1 16:40:38 2022] POST /identity/v3/users => generated 283 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 16:40:38.644695 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3609ad2-65f1-4da7-9613-842d4ceb6225 admin admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:40:38.644954 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-f6985d35-5dac-401f-814d-74006728376e None None] MFA Rules not processed for user `ea27a708e2c84ccc905ae783524bd12f`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:40:38.648535 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-f6985d35-5dac-401f-814d-74006728376e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:40:38.664652 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3609ad2-65f1-4da7-9613-842d4ceb6225 admin admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:40:38.670205 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 877/1768] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 1 16:40:38 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 16:40:38.671261 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6985d35-5dac-401f-814d-74006728376e None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:40:38.682991 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fded3034-d3fe-4584-8fa8-172c42b04a59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:40:38.684778 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fded3034-d3fe-4584-8fa8-172c42b04a59 admin admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:40:38.685439 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fded3034-d3fe-4584-8fa8-172c42b04a59 admin admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:40:38.686503 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fded3034-d3fe-4584-8fa8-172c42b04a59 admin admin] PATH_INFO: `/v3/roles` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:40:38.688117 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fded3034-d3fe-4584-8fa8-172c42b04a59 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:40:38.692925 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fded3034-d3fe-4584-8fa8-172c42b04a59 admin admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:40:38.702892 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6985d35-5dac-401f-814d-74006728376e None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:40:38.723404 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be55e8c5-77ad-401f-8687-67ae1a890b92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:40:38.724266 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be55e8c5-77ad-401f-8687-67ae1a890b92 admin admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:40:38.724697 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be55e8c5-77ad-401f-8687-67ae1a890b92 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:40:38.725069 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be55e8c5-77ad-401f-8687-67ae1a890b92 admin admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:40:38.726478 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be55e8c5-77ad-401f-8687-67ae1a890b92 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:40:38.727077 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 892/1769] 10.222.0.57 () {58 vars in 1121 bytes} [Fri Jul 1 16:40:38 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 16:40:38.730822 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fded3034-d3fe-4584-8fa8-172c42b04a59 admin admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:40:38.730959 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be55e8c5-77ad-401f-8687-67ae1a890b92 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:40:38.733855 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-fded3034-d3fe-4584-8fa8-172c42b04a59 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:40:38.761200 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 878/1770] 10.222.0.57 () {58 vars in 1121 bytes} [Fri Jul 1 16:40:38 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 16:40:38.766050 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be55e8c5-77ad-401f-8687-67ae1a890b92 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:40:38.789990 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be55e8c5-77ad-401f-8687-67ae1a890b92 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:40:38.805601 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19569b60-85fe-444f-8998-84376f526579 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:40:38.806685 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19569b60-85fe-444f-8998-84376f526579 admin admin] REQUEST_METHOD: `PUT` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:40:38.807065 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19569b60-85fe-444f-8998-84376f526579 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:40:38.807438 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19569b60-85fe-444f-8998-84376f526579 admin admin] PATH_INFO: `/v3/projects/811c93ee3a4849c5974599f1a03f9188/users/6a5d2d3d9b004b4f91d51609d04b40d7/roles/84ee739146934c1ab32c501ff8093644` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:40:38.856087 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19569b60-85fe-444f-8998-84376f526579 admin admin] RBAC: Authorizing `identity:create_grant(project_id=811c93ee3a4849c5974599f1a03f9188, user_id=6a5d2d3d9b004b4f91d51609d04b40d7, role_id=84ee739146934c1ab32c501ff8093644)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:40:38.863811 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19569b60-85fe-444f-8998-84376f526579 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:40:38.877254 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:40:38.888165 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 879/1771] 10.222.0.57 () {60 vars in 1482 bytes} [Fri Jul 1 16:40:38 2022] PUT /identity/v3/projects/811c93ee3a4849c5974599f1a03f9188/users/6a5d2d3d9b004b4f91d51609d04b40d7/roles/84ee739146934c1ab32c501ff8093644 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 16:40:38.889816 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19569b60-85fe-444f-8998-84376f526579 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:40:38.964134 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fded3034-d3fe-4584-8fa8-172c42b04a59 admin admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:40:39.111299 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19569b60-85fe-444f-8998-84376f526579 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:40:39.131203 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab81912-fe63-4151-804b-5f6b61675257 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:40:39.132146 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab81912-fe63-4151-804b-5f6b61675257 admin admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:40:39.132537 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab81912-fe63-4151-804b-5f6b61675257 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:40:39.132922 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab81912-fe63-4151-804b-5f6b61675257 admin admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:40:39.135386 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab81912-fe63-4151-804b-5f6b61675257 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:40:39.137210 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab81912-fe63-4151-804b-5f6b61675257 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:40:39.140966 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:40:39.172374 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 880/1772] 10.222.0.57 () {58 vars in 1121 bytes} [Fri Jul 1 16:40:38 2022] GET /identity/v3/roles => generated 1420 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 16:40:39.174197 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab81912-fe63-4151-804b-5f6b61675257 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:40:39.218444 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6472c63-40c8-4dcd-b3ac-46ec41d9aa43 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a50f745b31941b7a7265c0fc852f48e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0788e6725a84e57a4174284c1845755', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:40:39.219038 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab81912-fe63-4151-804b-5f6b61675257 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:40:39.219848 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6472c63-40c8-4dcd-b3ac-46ec41d9aa43 service nova] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:40:39.224082 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6472c63-40c8-4dcd-b3ac-46ec41d9aa43 service nova] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:40:39.224755 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6472c63-40c8-4dcd-b3ac-46ec41d9aa43 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:40:39.233883 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d924e64-e12b-49df-b8cd-5b853ca77b3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:40:39.235220 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d924e64-e12b-49df-b8cd-5b853ca77b3a admin admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:40:39.239791 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d924e64-e12b-49df-b8cd-5b853ca77b3a admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:40:39.240422 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d924e64-e12b-49df-b8cd-5b853ca77b3a admin admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:40:39.244265 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d924e64-e12b-49df-b8cd-5b853ca77b3a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:40:39.250721 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d924e64-e12b-49df-b8cd-5b853ca77b3a admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:40:39.301904 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 881/1773] 10.222.0.57 () {58 vars in 1121 bytes} [Fri Jul 1 16:40:39 2022] GET /identity/v3/roles => generated 1420 bytes in 126 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 16:40:39.301904 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569e2fb4-32c8-4885-a90b-4b290cfa4430 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:40:39.301904 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569e2fb4-32c8-4885-a90b-4b290cfa4430 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:40:39.301904 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569e2fb4-32c8-4885-a90b-4b290cfa4430 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:40:39.337891 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-569e2fb4-32c8-4885-a90b-4b290cfa4430 None None] MFA Rules not processed for user `c4de1b1eeaed4fc5ae7200b2c7a90957`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:40:39.341123 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-569e2fb4-32c8-4885-a90b-4b290cfa4430 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:40:39.399351 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6472c63-40c8-4dcd-b3ac-46ec41d9aa43 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:40:39.403343 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6472c63-40c8-4dcd-b3ac-46ec41d9aa43 service nova] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:40:39.447805 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 893/1774] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 1 16:40:38 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 719 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 16:40:39.449245 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6472c63-40c8-4dcd-b3ac-46ec41d9aa43 service nova] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:40:39.475818 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6472c63-40c8-4dcd-b3ac-46ec41d9aa43 service nova] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:40:39.490109 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130f8c68-9a78-4484-9f83-b555a7b80d81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:40:39.491202 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130f8c68-9a78-4484-9f83-b555a7b80d81 admin admin] REQUEST_METHOD: `PUT` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:40:39.491491 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130f8c68-9a78-4484-9f83-b555a7b80d81 admin admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:40:39.491772 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130f8c68-9a78-4484-9f83-b555a7b80d81 admin admin] PATH_INFO: `/v3/projects/37c1044c791040508147811f544e2665/users/00ab33726a1b4cfda4b49a0bb4f88348/roles/983be40bdab04c478eaa90c6b6a46b14` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:40:39.495164 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 882/1775] 10.222.0.57 () {58 vars in 1009 bytes} [Fri Jul 1 16:40:39 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 16:40:39.499052 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569e2fb4-32c8-4885-a90b-4b290cfa4430 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:40:39.517694 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-130f8c68-9a78-4484-9f83-b555a7b80d81 admin admin] RBAC: Authorizing `identity:create_grant(project_id=37c1044c791040508147811f544e2665, user_id=00ab33726a1b4cfda4b49a0bb4f88348, role_id=983be40bdab04c478eaa90c6b6a46b14)` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:40:39.522613 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-130f8c68-9a78-4484-9f83-b555a7b80d81 admin admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:40:39.532857 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569e2fb4-32c8-4885-a90b-4b290cfa4430 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:40:39.554421 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 894/1776] 10.222.0.57 () {60 vars in 1482 bytes} [Fri Jul 1 16:40:39 2022] PUT /identity/v3/projects/37c1044c791040508147811f544e2665/users/00ab33726a1b4cfda4b49a0bb4f88348/roles/983be40bdab04c478eaa90c6b6a46b14 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 16:40:39.556080 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130f8c68-9a78-4484-9f83-b555a7b80d81 admin admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:40:39.563963 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49301795-9530-40bb-b794-d19d9b334c8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:40:39.564596 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-130f8c68-9a78-4484-9f83-b555a7b80d81 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:40:39.566566 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49301795-9530-40bb-b794-d19d9b334c8b admin admin] REQUEST_METHOD: `PUT` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:40:39.568721 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49301795-9530-40bb-b794-d19d9b334c8b admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:40:39.569282 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49301795-9530-40bb-b794-d19d9b334c8b admin admin] PATH_INFO: `/v3/projects/811c93ee3a4849c5974599f1a03f9188/users/6a5d2d3d9b004b4f91d51609d04b40d7/roles/84ee739146934c1ab32c501ff8093644` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:40:39.575138 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49301795-9530-40bb-b794-d19d9b334c8b admin admin] RBAC: Authorizing `identity:create_grant(project_id=811c93ee3a4849c5974599f1a03f9188, user_id=6a5d2d3d9b004b4f91d51609d04b40d7, role_id=84ee739146934c1ab32c501ff8093644)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:40:39.586065 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49301795-9530-40bb-b794-d19d9b334c8b admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:40:39.623397 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 883/1777] 10.222.0.57 () {60 vars in 1482 bytes} [Fri Jul 1 16:40:39 2022] PUT /identity/v3/projects/811c93ee3a4849c5974599f1a03f9188/users/6a5d2d3d9b004b4f91d51609d04b40d7/roles/84ee739146934c1ab32c501ff8093644 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 16:40:39.627673 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6920450-2149-4554-b152-30cf453e2f7f None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:40:39.628602 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6920450-2149-4554-b152-30cf453e2f7f None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:40:39.629496 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6920450-2149-4554-b152-30cf453e2f7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:40:39.701665 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-e6920450-2149-4554-b152-30cf453e2f7f None None] MFA Rules not processed for user `00ab33726a1b4cfda4b49a0bb4f88348`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:40:39.703038 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-e6920450-2149-4554-b152-30cf453e2f7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:40:39.705217 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:40:39.728333 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130f8c68-9a78-4484-9f83-b555a7b80d81 admin admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:40:39.867838 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 884/1778] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 1 16:40:39 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 243 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 16:40:39.869373 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f4baab-5fc7-4fc5-a4c9-0bcb06703786 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:40:39.869951 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f4baab-5fc7-4fc5-a4c9-0bcb06703786 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:40:39.870381 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f4baab-5fc7-4fc5-a4c9-0bcb06703786 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:40:39.985919 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328c4800-5427-4050-afb3-8462b2d1f96c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:40:39.986099 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-e0f4baab-5fc7-4fc5-a4c9-0bcb06703786 None None] MFA Rules not processed for user `6a5d2d3d9b004b4f91d51609d04b40d7`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:40:39.989644 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328c4800-5427-4050-afb3-8462b2d1f96c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:40:39.990951 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-e0f4baab-5fc7-4fc5-a4c9-0bcb06703786 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:40:39.992365 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328c4800-5427-4050-afb3-8462b2d1f96c admin admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:40:39.992668 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328c4800-5427-4050-afb3-8462b2d1f96c admin admin] PATH_INFO: `/v3/projects/d07b741a7f8e42c8b7d0c72ef29b6be1` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:40:39.994230 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-328c4800-5427-4050-afb3-8462b2d1f96c admin admin] RBAC: Authorizing `identity:delete_project(project_id=d07b741a7f8e42c8b7d0c72ef29b6be1)` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:40:40.008820 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-328c4800-5427-4050-afb3-8462b2d1f96c admin admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:40:40.107863 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:40:40.137560 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-328c4800-5427-4050-afb3-8462b2d1f96c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd07b741a7f8e42c8b7d0c72ef29b6be1', 'request_id': 'req-328c4800-5427-4050-afb3-8462b2d1f96c'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:40:40.159626 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-328c4800-5427-4050-afb3-8462b2d1f96c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd07b741a7f8e42c8b7d0c72ef29b6be1', 'request_id': 'req-328c4800-5427-4050-afb3-8462b2d1f96c'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:40:40.171273 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 885/1779] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 1 16:40:39 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 303 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 16:40:40.175763 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42529f9d-1a5e-4f5b-87e2-688eb747e0f6 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:40:40.176203 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42529f9d-1a5e-4f5b-87e2-688eb747e0f6 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:40:40.176584 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42529f9d-1a5e-4f5b-87e2-688eb747e0f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:40:40.180017 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-328c4800-5427-4050-afb3-8462b2d1f96c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd07b741a7f8e42c8b7d0c72ef29b6be1', 'request_id': 'req-328c4800-5427-4050-afb3-8462b2d1f96c'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:40:40.196192 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-328c4800-5427-4050-afb3-8462b2d1f96c admin admin] The token cache is being invalidate because project d07b741a7f8e42c8b7d0c72ef29b6be1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80312) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 01 16:40:40.196383 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-328c4800-5427-4050-afb3-8462b2d1f96c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:40:40.196984 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-328c4800-5427-4050-afb3-8462b2d1f96c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:40:40.199714 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 895/1780] 10.222.0.57 () {58 vars in 1232 bytes} [Fri Jul 1 16:40:39 2022] DELETE /identity/v3/projects/d07b741a7f8e42c8b7d0c72ef29b6be1 => generated 0 bytes in 645 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 16:40:40.203391 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328c4800-5427-4050-afb3-8462b2d1f96c admin admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:40:40.206775 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-328c4800-5427-4050-afb3-8462b2d1f96c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:40:40.242506 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-42529f9d-1a5e-4f5b-87e2-688eb747e0f6 None None] MFA Rules not processed for user `71c383c04f9543d8ab97a6c0db268ca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:40:40.243894 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-42529f9d-1a5e-4f5b-87e2-688eb747e0f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:40:40.302798 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328c4800-5427-4050-afb3-8462b2d1f96c admin admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:40:40.311117 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86d6d61-9fcb-4508-96f4-e315184ba02d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '859fe410c2d2450fb803c6244839f27c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0788e6725a84e57a4174284c1845755', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:40:40.312267 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86d6d61-9fcb-4508-96f4-e315184ba02d service swift] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:40:40.312744 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86d6d61-9fcb-4508-96f4-e315184ba02d service swift] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:40:40.313748 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86d6d61-9fcb-4508-96f4-e315184ba02d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:40:40.317194 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-e86d6d61-9fcb-4508-96f4-e315184ba02d service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:40:40.412534 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:40:40.425214 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 886/1781] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 1 16:40:40 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 253 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 16:40:40.427552 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42529f9d-1a5e-4f5b-87e2-688eb747e0f6 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:40:40.430540 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-42529f9d-1a5e-4f5b-87e2-688eb747e0f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:40:40.469520 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e86d6d61-9fcb-4508-96f4-e315184ba02d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:40:40.473080 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e86d6d61-9fcb-4508-96f4-e315184ba02d service swift] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:40:40.483935 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 896/1782] 10.222.0.57 () {60 vars in 1384 bytes} [Fri Jul 1 16:40:40 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 284 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 01 16:40:40.484936 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86d6d61-9fcb-4508-96f4-e315184ba02d service swift] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:40:40.486442 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-e86d6d61-9fcb-4508-96f4-e315184ba02d service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:40:40.565091 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86d6d61-9fcb-4508-96f4-e315184ba02d service swift] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:40:40.570572 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7e38cd-5ca2-492b-bddc-56a577a66f4b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbddf971a2354bbdbb9da1d63f3abf88', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0788e6725a84e57a4174284c1845755', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:40:40.571569 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7e38cd-5ca2-492b-bddc-56a577a66f4b service cinder] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:40:40.572044 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7e38cd-5ca2-492b-bddc-56a577a66f4b service cinder] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:40:40.572407 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7e38cd-5ca2-492b-bddc-56a577a66f4b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:40:40.575126 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-2f7e38cd-5ca2-492b-bddc-56a577a66f4b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:40:40.685422 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f7e38cd-5ca2-492b-bddc-56a577a66f4b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:40:40.688826 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f7e38cd-5ca2-492b-bddc-56a577a66f4b service cinder] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:40:40.695473 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42529f9d-1a5e-4f5b-87e2-688eb747e0f6 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:40:40.713386 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce48488-2bab-4184-ae06-ce2a4d8c83aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:40:40.716847 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce48488-2bab-4184-ae06-ce2a4d8c83aa admin admin] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:40:40.717147 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 897/1783] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 1 16:40:40 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 231 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 16:40:40.717430 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce48488-2bab-4184-ae06-ce2a4d8c83aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:40:40.717728 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce48488-2bab-4184-ae06-ce2a4d8c83aa admin admin] PATH_INFO: `/v3/projects` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:40:40.718489 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce48488-2bab-4184-ae06-ce2a4d8c83aa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:40:40.720306 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7e38cd-5ca2-492b-bddc-56a577a66f4b service cinder] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:40:40.734244 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce48488-2bab-4184-ae06-ce2a4d8c83aa admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:40:40.737029 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-2f7e38cd-5ca2-492b-bddc-56a577a66f4b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:40:40.780441 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7e38cd-5ca2-492b-bddc-56a577a66f4b service cinder] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:40:40.794075 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e506792e-d268-44b5-92ce-da269c1f13eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:40:40.796563 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e506792e-d268-44b5-92ce-da269c1f13eb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:40:40.797151 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e506792e-d268-44b5-92ce-da269c1f13eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:40:40.798658 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e506792e-d268-44b5-92ce-da269c1f13eb admin admin] PATH_INFO: `/v3/users/15e12b51d19c451d91be7bcecc6b66a1` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:40:40.803778 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e506792e-d268-44b5-92ce-da269c1f13eb admin admin] RBAC: Authorizing `identity:delete_user(user_id=15e12b51d19c451d91be7bcecc6b66a1)` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:40:40.807172 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e506792e-d268-44b5-92ce-da269c1f13eb admin admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:40:40.819497 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 887/1784] 10.222.0.57 () {60 vars in 1152 bytes} [Fri Jul 1 16:40:40 2022] POST /identity/v3/projects => generated 363 bytes in 393 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 16:40:40.820944 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1258fa-d009-4763-a62f-87ebf945bc09 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:40:40.821354 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1258fa-d009-4763-a62f-87ebf945bc09 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:40:40.821731 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1258fa-d009-4763-a62f-87ebf945bc09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:40:40.902936 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-8f1258fa-d009-4763-a62f-87ebf945bc09 None None] MFA Rules not processed for user `71c383c04f9543d8ab97a6c0db268ca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:40:40.910886 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-8f1258fa-d009-4763-a62f-87ebf945bc09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:40:40.962766 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-e506792e-d268-44b5-92ce-da269c1f13eb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '15e12b51d19c451d91be7bcecc6b66a1'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:40:40.993640 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-e506792e-d268-44b5-92ce-da269c1f13eb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '15e12b51d19c451d91be7bcecc6b66a1'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:40:41.013376 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-e506792e-d268-44b5-92ce-da269c1f13eb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '15e12b51d19c451d91be7bcecc6b66a1'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:40:41.015317 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-e506792e-d268-44b5-92ce-da269c1f13eb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '15e12b51d19c451d91be7bcecc6b66a1'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:40:41.017947 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-e506792e-d268-44b5-92ce-da269c1f13eb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '15e12b51d19c451d91be7bcecc6b66a1'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:40:41.024115 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:40:41.058070 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 898/1785] 10.222.0.57 () {58 vars in 1223 bytes} [Fri Jul 1 16:40:40 2022] DELETE /identity/v3/users/15e12b51d19c451d91be7bcecc6b66a1 => generated 0 bytes in 340 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 16:40:41.072001 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e506792e-d268-44b5-92ce-da269c1f13eb admin admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:40:41.077578 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-e506792e-d268-44b5-92ce-da269c1f13eb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:40:41.157314 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 888/1786] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 1 16:40:40 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 337 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 16:40:41.161225 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f1258fa-d009-4763-a62f-87ebf945bc09 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:40:41.170168 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-8f1258fa-d009-4763-a62f-87ebf945bc09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:40:41.234326 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:40:41.331630 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e506792e-d268-44b5-92ce-da269c1f13eb admin admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:40:41.347238 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af04053e-fd33-49f9-a92a-b8a0c32b0ea3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:40:41.351406 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af04053e-fd33-49f9-a92a-b8a0c32b0ea3 admin admin] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:40:41.352037 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af04053e-fd33-49f9-a92a-b8a0c32b0ea3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:40:41.352562 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af04053e-fd33-49f9-a92a-b8a0c32b0ea3 admin admin] PATH_INFO: `/v3/users` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:40:41.353956 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af04053e-fd33-49f9-a92a-b8a0c32b0ea3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:40:41.360022 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af04053e-fd33-49f9-a92a-b8a0c32b0ea3 admin admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:40:41.436970 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 899/1787] 10.222.0.57 () {60 vars in 1143 bytes} [Fri Jul 1 16:40:41 2022] POST /identity/v3/users => generated 288 bytes in 378 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 16:40:41.437798 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af04053e-fd33-49f9-a92a-b8a0c32b0ea3 admin admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:40:41.450911 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-af04053e-fd33-49f9-a92a-b8a0c32b0ea3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:40:41.465338 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f1258fa-d009-4763-a62f-87ebf945bc09 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:40:41.760940 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb949f9-d1e1-4150-a76f-37d645c0c944 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbddf971a2354bbdbb9da1d63f3abf88', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0788e6725a84e57a4174284c1845755', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:40:41.761759 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb949f9-d1e1-4150-a76f-37d645c0c944 service cinder] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:40:41.768870 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb949f9-d1e1-4150-a76f-37d645c0c944 service cinder] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:40:41.771181 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb949f9-d1e1-4150-a76f-37d645c0c944 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:40:41.789097 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-5eb949f9-d1e1-4150-a76f-37d645c0c944 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:40:41.844153 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af04053e-fd33-49f9-a92a-b8a0c32b0ea3 admin admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:40:41.870227 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac37812-3bf5-48f0-b15c-576d318b88e6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a50f745b31941b7a7265c0fc852f48e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0788e6725a84e57a4174284c1845755', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:40:41.872609 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac37812-3bf5-48f0-b15c-576d318b88e6 service nova] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:40:41.873005 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac37812-3bf5-48f0-b15c-576d318b88e6 service nova] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:40:41.873945 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac37812-3bf5-48f0-b15c-576d318b88e6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:40:41.884098 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-5ac37812-3bf5-48f0-b15c-576d318b88e6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:40:41.976912 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb949f9-d1e1-4150-a76f-37d645c0c944 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:40:41.979016 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb949f9-d1e1-4150-a76f-37d645c0c944 service cinder] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:40:42.008458 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 889/1788] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 1 16:40:41 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 851 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 16:40:42.010565 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb949f9-d1e1-4150-a76f-37d645c0c944 service cinder] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:40:42.045460 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb949f9-d1e1-4150-a76f-37d645c0c944 service cinder] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:40:42.063677 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81218624-38eb-427d-964a-03d24978449b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:40:42.070230 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81218624-38eb-427d-964a-03d24978449b admin admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:40:42.072157 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81218624-38eb-427d-964a-03d24978449b admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:40:42.073269 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81218624-38eb-427d-964a-03d24978449b admin admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:40:42.074928 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81218624-38eb-427d-964a-03d24978449b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:40:42.083241 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81218624-38eb-427d-964a-03d24978449b admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:40:42.145203 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 890/1789] 10.222.0.57 () {58 vars in 1121 bytes} [Fri Jul 1 16:40:42 2022] GET /identity/v3/roles => generated 1420 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 16:40:42.148167 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81218624-38eb-427d-964a-03d24978449b admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:40:42.156236 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac37812-3bf5-48f0-b15c-576d318b88e6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:40:42.163766 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac37812-3bf5-48f0-b15c-576d318b88e6 service nova] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:40:42.196362 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81218624-38eb-427d-964a-03d24978449b admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:40:42.233620 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 900/1790] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 1 16:40:41 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 796 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 16:40:42.238065 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4abf61db-4528-4bd1-b39f-faa1a18963e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:40:42.241005 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abf61db-4528-4bd1-b39f-faa1a18963e5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:40:42.255733 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abf61db-4528-4bd1-b39f-faa1a18963e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:40:42.256076 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abf61db-4528-4bd1-b39f-faa1a18963e5 admin admin] PATH_INFO: `/v3/projects/52a7aaae39084cdf84c1044da9627eda/users/16624875454e4a39a86aeeebcd1bb874/roles/84ee739146934c1ab32c501ff8093644` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:40:42.304340 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4abf61db-4528-4bd1-b39f-faa1a18963e5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=52a7aaae39084cdf84c1044da9627eda, user_id=16624875454e4a39a86aeeebcd1bb874, role_id=84ee739146934c1ab32c501ff8093644)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:40:42.310938 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4abf61db-4528-4bd1-b39f-faa1a18963e5 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:40:42.343546 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 891/1791] 10.222.0.57 () {60 vars in 1482 bytes} [Fri Jul 1 16:40:42 2022] PUT /identity/v3/projects/52a7aaae39084cdf84c1044da9627eda/users/16624875454e4a39a86aeeebcd1bb874/roles/84ee739146934c1ab32c501ff8093644 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 16:40:42.344890 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac37812-3bf5-48f0-b15c-576d318b88e6 service nova] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:40:42.364156 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac37812-3bf5-48f0-b15c-576d318b88e6 service nova] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:40:42.414162 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:DELETE=1,keystone:SELECT=96,keystone:INSERT=3 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:40:42.450087 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4abf61db-4528-4bd1-b39f-faa1a18963e5 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:40:42.453228 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-4abf61db-4528-4bd1-b39f-faa1a18963e5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:40:42.624921 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:40:42.654454 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5202e31c-af6c-44cf-b693-57036d26255e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:40:42.654454 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5202e31c-af6c-44cf-b693-57036d26255e admin admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:40:42.654454 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5202e31c-af6c-44cf-b693-57036d26255e admin admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:40:42.654454 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5202e31c-af6c-44cf-b693-57036d26255e admin admin] PATH_INFO: `/v3/roles` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:40:42.655132 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5202e31c-af6c-44cf-b693-57036d26255e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:40:42.658079 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5202e31c-af6c-44cf-b693-57036d26255e admin admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:40:42.670820 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4abf61db-4528-4bd1-b39f-faa1a18963e5 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:40:42.677115 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550ffb44-b6c2-4ac6-8d6f-591e1c68aaa6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:40:42.679067 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550ffb44-b6c2-4ac6-8d6f-591e1c68aaa6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:40:42.679449 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550ffb44-b6c2-4ac6-8d6f-591e1c68aaa6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:40:42.680278 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550ffb44-b6c2-4ac6-8d6f-591e1c68aaa6 admin admin] PATH_INFO: `/v3/projects/53eba863d47f4e669ef367658a83a96b` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:40:42.682095 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-550ffb44-b6c2-4ac6-8d6f-591e1c68aaa6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=53eba863d47f4e669ef367658a83a96b)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:40:42.684789 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-550ffb44-b6c2-4ac6-8d6f-591e1c68aaa6 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:40:42.689550 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 901/1792] 10.222.0.57 () {58 vars in 1121 bytes} [Fri Jul 1 16:40:42 2022] GET /identity/v3/roles => generated 1420 bytes in 345 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 16:40:42.694387 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5202e31c-af6c-44cf-b693-57036d26255e admin admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:40:42.725230 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5202e31c-af6c-44cf-b693-57036d26255e admin admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:40:42.742737 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2f9671-29d1-4f6b-a6e9-79df7080f8b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:40:42.744069 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2f9671-29d1-4f6b-a6e9-79df7080f8b2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:40:42.744539 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2f9671-29d1-4f6b-a6e9-79df7080f8b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:40:42.744998 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2f9671-29d1-4f6b-a6e9-79df7080f8b2 admin admin] PATH_INFO: `/v3/projects/52a7aaae39084cdf84c1044da9627eda/users/16624875454e4a39a86aeeebcd1bb874/roles/84ee739146934c1ab32c501ff8093644` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:40:42.755329 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b2f9671-29d1-4f6b-a6e9-79df7080f8b2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=52a7aaae39084cdf84c1044da9627eda, user_id=16624875454e4a39a86aeeebcd1bb874, role_id=84ee739146934c1ab32c501ff8093644)` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:40:42.758467 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b2f9671-29d1-4f6b-a6e9-79df7080f8b2 admin admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:40:42.812529 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 902/1793] 10.222.0.57 () {60 vars in 1482 bytes} [Fri Jul 1 16:40:42 2022] PUT /identity/v3/projects/52a7aaae39084cdf84c1044da9627eda/users/16624875454e4a39a86aeeebcd1bb874/roles/84ee739146934c1ab32c501ff8093644 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 16:40:42.823657 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28037895-931d-4c42-9a53-05e26ca4c4fd None None] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:40:42.823947 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28037895-931d-4c42-9a53-05e26ca4c4fd None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:40:42.824233 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28037895-931d-4c42-9a53-05e26ca4c4fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:40:42.894000 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-550ffb44-b6c2-4ac6-8d6f-591e1c68aaa6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '53eba863d47f4e669ef367658a83a96b', 'request_id': 'req-550ffb44-b6c2-4ac6-8d6f-591e1c68aaa6'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:40:42.908074 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.auth.core [None req-28037895-931d-4c42-9a53-05e26ca4c4fd None None] MFA Rules not processed for user `16624875454e4a39a86aeeebcd1bb874`. Rule list: `[]` (Enabled: `True`). {{(pid=80312) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:40:42.910211 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-28037895-931d-4c42-9a53-05e26ca4c4fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:40:42.921784 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-550ffb44-b6c2-4ac6-8d6f-591e1c68aaa6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '53eba863d47f4e669ef367658a83a96b', 'request_id': 'req-550ffb44-b6c2-4ac6-8d6f-591e1c68aaa6'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:40:42.932896 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-550ffb44-b6c2-4ac6-8d6f-591e1c68aaa6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '53eba863d47f4e669ef367658a83a96b', 'request_id': 'req-550ffb44-b6c2-4ac6-8d6f-591e1c68aaa6'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:40:43.004727 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-550ffb44-b6c2-4ac6-8d6f-591e1c68aaa6 admin admin] The token cache is being invalidate because project 53eba863d47f4e669ef367658a83a96b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80311) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 01 16:40:43.005151 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-550ffb44-b6c2-4ac6-8d6f-591e1c68aaa6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:40:43.008298 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-550ffb44-b6c2-4ac6-8d6f-591e1c68aaa6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:40:43.014093 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 892/1794] 10.222.0.57 () {58 vars in 1232 bytes} [Fri Jul 1 16:40:42 2022] DELETE /identity/v3/projects/53eba863d47f4e669ef367658a83a96b => generated 0 bytes in 566 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 01 16:40:43.020784 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a50994a-b586-4a57-b301-3b73ff0473d1 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:40:43.021074 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a50994a-b586-4a57-b301-3b73ff0473d1 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:40:43.021358 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a50994a-b586-4a57-b301-3b73ff0473d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:40:43.122272 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-8a50994a-b586-4a57-b301-3b73ff0473d1 None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:40:43.126426 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-8a50994a-b586-4a57-b301-3b73ff0473d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:40:43.174516 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 903/1795] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 1 16:40:42 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 350 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 01 16:40:43.184336 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28037895-931d-4c42-9a53-05e26ca4c4fd None None] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:40:43.185763 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-28037895-931d-4c42-9a53-05e26ca4c4fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:40:43.371099 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28037895-931d-4c42-9a53-05e26ca4c4fd None None] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:40:43.383511 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5510f69d-d678-44d3-9991-79d8fe1f6ce0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbddf971a2354bbdbb9da1d63f3abf88', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0788e6725a84e57a4174284c1845755', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:40:43.384277 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5510f69d-d678-44d3-9991-79d8fe1f6ce0 service cinder] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:40:43.384567 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5510f69d-d678-44d3-9991-79d8fe1f6ce0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:40:43.384834 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5510f69d-d678-44d3-9991-79d8fe1f6ce0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:40:43.405203 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 893/1796] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 1 16:40:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 386 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 16:40:43.408712 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62f61f6-d941-43f0-9e06-d154941ec3ec None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:40:43.412438 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62f61f6-d941-43f0-9e06-d154941ec3ec None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:40:43.415382 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62f61f6-d941-43f0-9e06-d154941ec3ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:40:43.425250 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5510f69d-d678-44d3-9991-79d8fe1f6ce0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:40:43.426820 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5510f69d-d678-44d3-9991-79d8fe1f6ce0 service cinder] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:40:43.452902 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 904/1797] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 1 16:40:43 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 269 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 01 16:40:43.494997 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-b62f61f6-d941-43f0-9e06-d154941ec3ec None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:40:43.496796 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-b62f61f6-d941-43f0-9e06-d154941ec3ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:40:43.519394 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 894/1798] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 1 16:40:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 16:40:43.524982 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5510f69d-d678-44d3-9991-79d8fe1f6ce0 service cinder] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:40:43.563257 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5510f69d-d678-44d3-9991-79d8fe1f6ce0 service cinder] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:40:43.585026 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d31a447-67f9-45f1-9ef5-0459c7eb17fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:40:43.590006 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d31a447-67f9-45f1-9ef5-0459c7eb17fe admin admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:40:43.590334 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d31a447-67f9-45f1-9ef5-0459c7eb17fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:40:43.593073 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d31a447-67f9-45f1-9ef5-0459c7eb17fe admin admin] PATH_INFO: `/v3/domains` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:40:43.595390 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d31a447-67f9-45f1-9ef5-0459c7eb17fe admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80312) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:40:43.595791 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d31a447-67f9-45f1-9ef5-0459c7eb17fe admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:40:43.597677 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d31a447-67f9-45f1-9ef5-0459c7eb17fe admin admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:40:43.623145 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 905/1799] 10.222.0.57 () {58 vars in 1152 bytes} [Fri Jul 1 16:40:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:40:43.630197 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62f61f6-d941-43f0-9e06-d154941ec3ec None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:40:43.649121 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62f61f6-d941-43f0-9e06-d154941ec3ec None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:40:43.655214 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d31a447-67f9-45f1-9ef5-0459c7eb17fe admin admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:40:43.657114 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-7d31a447-67f9-45f1-9ef5-0459c7eb17fe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:40:43.686492 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34862e9-f8bb-4d5f-9000-301c144aa6d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:40:43.692551 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34862e9-f8bb-4d5f-9000-301c144aa6d1 admin admin] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:40:43.695572 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34862e9-f8bb-4d5f-9000-301c144aa6d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:40:43.696800 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34862e9-f8bb-4d5f-9000-301c144aa6d1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:40:43.698474 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f34862e9-f8bb-4d5f-9000-301c144aa6d1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:40:43.699924 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d31a447-67f9-45f1-9ef5-0459c7eb17fe admin admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:40:43.704313 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f34862e9-f8bb-4d5f-9000-301c144aa6d1 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:40:43.723544 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99948476-777c-4adb-b5f4-c50e8f1f4270 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbddf971a2354bbdbb9da1d63f3abf88', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0788e6725a84e57a4174284c1845755', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:40:43.724315 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99948476-777c-4adb-b5f4-c50e8f1f4270 service cinder] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:40:43.724315 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99948476-777c-4adb-b5f4-c50e8f1f4270 service cinder] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:40:43.724723 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99948476-777c-4adb-b5f4-c50e8f1f4270 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:40:43.743634 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-99948476-777c-4adb-b5f4-c50e8f1f4270 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:40:43.792905 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 895/1800] 10.222.0.57 () {60 vars in 1152 bytes} [Fri Jul 1 16:40:43 2022] POST /identity/v3/projects => generated 385 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 16:40:43.794332 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34862e9-f8bb-4d5f-9000-301c144aa6d1 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:40:43.812066 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:40:43.939719 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:40:43.977648 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34862e9-f8bb-4d5f-9000-301c144aa6d1 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:40:43.983432 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf124cb-8f9c-4d13-b587-a475847d1938 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:40:43.984193 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf124cb-8f9c-4d13-b587-a475847d1938 admin admin] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:40:43.984481 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf124cb-8f9c-4d13-b587-a475847d1938 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:40:43.984764 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf124cb-8f9c-4d13-b587-a475847d1938 admin admin] PATH_INFO: `/v3/users` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:40:43.986997 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf124cb-8f9c-4d13-b587-a475847d1938 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:40:43.990361 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99948476-777c-4adb-b5f4-c50e8f1f4270 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:40:43.991981 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf124cb-8f9c-4d13-b587-a475847d1938 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:40:43.995443 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99948476-777c-4adb-b5f4-c50e8f1f4270 service cinder] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:40:44.038846 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 896/1801] 10.222.0.57 () {60 vars in 1143 bytes} [Fri Jul 1 16:40:43 2022] POST /identity/v3/users => generated 293 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 16:40:44.039852 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf124cb-8f9c-4d13-b587-a475847d1938 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:40:44.055191 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 906/1802] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 1 16:40:43 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 400 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 01 16:40:44.072439 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf124cb-8f9c-4d13-b587-a475847d1938 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:40:44.084740 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e8b07a-02aa-4962-b290-f41ac2dbc42a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:40:44.085556 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e8b07a-02aa-4962-b290-f41ac2dbc42a admin admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:40:44.085802 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e8b07a-02aa-4962-b290-f41ac2dbc42a admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:40:44.086330 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e8b07a-02aa-4962-b290-f41ac2dbc42a admin admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:40:44.087457 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e8b07a-02aa-4962-b290-f41ac2dbc42a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:40:44.094182 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e8b07a-02aa-4962-b290-f41ac2dbc42a admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:40:44.121971 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 897/1803] 10.222.0.57 () {58 vars in 1121 bytes} [Fri Jul 1 16:40:44 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 16:40:44.126137 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99948476-777c-4adb-b5f4-c50e8f1f4270 service cinder] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:40:44.144759 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99948476-777c-4adb-b5f4-c50e8f1f4270 service cinder] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:40:44.159844 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a327f20-7759-4673-8a48-37e4512cf22f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:40:44.160571 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a327f20-7759-4673-8a48-37e4512cf22f admin admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:40:44.160901 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a327f20-7759-4673-8a48-37e4512cf22f admin admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:40:44.161592 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a327f20-7759-4673-8a48-37e4512cf22f admin admin] PATH_INFO: `/v3/roles` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:40:44.164259 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a327f20-7759-4673-8a48-37e4512cf22f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:40:44.171463 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a327f20-7759-4673-8a48-37e4512cf22f admin admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:40:44.200551 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e8b07a-02aa-4962-b290-f41ac2dbc42a admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:40:44.201741 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 907/1804] 10.222.0.57 () {58 vars in 1121 bytes} [Fri Jul 1 16:40:44 2022] GET /identity/v3/roles => generated 1420 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 16:40:44.216131 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e8b07a-02aa-4962-b290-f41ac2dbc42a admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:40:44.231605 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec54323-455e-48a8-acdc-69cef3783735 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:40:44.237513 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec54323-455e-48a8-acdc-69cef3783735 admin admin] REQUEST_METHOD: `PUT` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:40:44.239676 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec54323-455e-48a8-acdc-69cef3783735 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:40:44.239987 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec54323-455e-48a8-acdc-69cef3783735 admin admin] PATH_INFO: `/v3/projects/725874a440d444e59ed18043b614dbee/users/bc263c83b4ac49a2a81f057c1ab031e4/roles/983be40bdab04c478eaa90c6b6a46b14` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:40:44.313900 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec54323-455e-48a8-acdc-69cef3783735 admin admin] RBAC: Authorizing `identity:create_grant(project_id=725874a440d444e59ed18043b614dbee, user_id=bc263c83b4ac49a2a81f057c1ab031e4, role_id=983be40bdab04c478eaa90c6b6a46b14)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:40:44.317076 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec54323-455e-48a8-acdc-69cef3783735 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:40:44.386829 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 898/1805] 10.222.0.57 () {60 vars in 1482 bytes} [Fri Jul 1 16:40:44 2022] PUT /identity/v3/projects/725874a440d444e59ed18043b614dbee/users/bc263c83b4ac49a2a81f057c1ab031e4/roles/983be40bdab04c478eaa90c6b6a46b14 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 16:40:44.398290 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e8f6b4-8704-4bff-a375-44b8e94ef66c None None] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:40:44.398916 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e8f6b4-8704-4bff-a375-44b8e94ef66c None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:40:44.399405 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e8f6b4-8704-4bff-a375-44b8e94ef66c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:40:44.479543 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.auth.core [None req-24e8f6b4-8704-4bff-a375-44b8e94ef66c None None] MFA Rules not processed for user `bc263c83b4ac49a2a81f057c1ab031e4`. Rule list: `[]` (Enabled: `True`). {{(pid=80312) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:40:44.480714 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-24e8f6b4-8704-4bff-a375-44b8e94ef66c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:40:44.723508 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec54323-455e-48a8-acdc-69cef3783735 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:40:44.730706 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 908/1806] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 1 16:40:44 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 333 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 16:40:44.948830 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec54323-455e-48a8-acdc-69cef3783735 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:40:44.957657 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05067ed-33e3-428f-b4e1-ff7cb58f8c8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:40:44.958688 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05067ed-33e3-428f-b4e1-ff7cb58f8c8d admin admin] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:40:44.959379 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05067ed-33e3-428f-b4e1-ff7cb58f8c8d admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:40:44.959821 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05067ed-33e3-428f-b4e1-ff7cb58f8c8d admin admin] PATH_INFO: `/v3/projects` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:40:44.960916 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f05067ed-33e3-428f-b4e1-ff7cb58f8c8d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:40:44.963886 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f05067ed-33e3-428f-b4e1-ff7cb58f8c8d admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:40:45.037874 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66629720-c68d-432b-8719-928aa473bd11 None None] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:40:45.038220 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66629720-c68d-432b-8719-928aa473bd11 None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:40:45.038517 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66629720-c68d-432b-8719-928aa473bd11 None None] PATH_INFO: `/` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:40:45.043951 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 909/1807] 10.222.0.57 () {54 vars in 912 bytes} [Fri Jul 1 16:40:45 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 01 16:40:45.057169 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66629720-c68d-432b-8719-928aa473bd11 None None] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:40:45.062149 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 899/1808] 10.222.0.57 () {60 vars in 1152 bytes} [Fri Jul 1 16:40:44 2022] POST /identity/v3/projects => generated 385 bytes in 341 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 16:40:45.066613 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c030667-d6d1-42e9-b4c3-ef05e1811a96 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:40:45.071796 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c030667-d6d1-42e9-b4c3-ef05e1811a96 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:40:45.073662 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c030667-d6d1-42e9-b4c3-ef05e1811a96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:40:45.164467 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-8c030667-d6d1-42e9-b4c3-ef05e1811a96 None None] MFA Rules not processed for user `4ed200ce29684c5f80354fa9a8638fe2`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:40:45.166213 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-8c030667-d6d1-42e9-b4c3-ef05e1811a96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:40:45.425312 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66629720-c68d-432b-8719-928aa473bd11 None None] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:40:45.455364 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348f5ddd-e39b-408f-a64e-e642644e8347 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:40:45.461660 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348f5ddd-e39b-408f-a64e-e642644e8347 admin admin] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:40:45.461954 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348f5ddd-e39b-408f-a64e-e642644e8347 admin admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:40:45.465627 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348f5ddd-e39b-408f-a64e-e642644e8347 admin admin] PATH_INFO: `/v3/users` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:40:45.469472 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-348f5ddd-e39b-408f-a64e-e642644e8347 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:40:45.482839 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-348f5ddd-e39b-408f-a64e-e642644e8347 admin admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:40:45.558599 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 900/1809] 10.222.0.57 () {58 vars in 1009 bytes} [Fri Jul 1 16:40:45 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 494 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 01 16:40:45.560198 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7052bd2-4aab-42f6-8d18-6496fa992ae5 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:40:45.560480 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7052bd2-4aab-42f6-8d18-6496fa992ae5 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:40:45.568914 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7052bd2-4aab-42f6-8d18-6496fa992ae5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:40:45.637093 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 910/1810] 10.222.0.57 () {60 vars in 1143 bytes} [Fri Jul 1 16:40:45 2022] POST /identity/v3/users => generated 299 bytes in 575 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 16:40:45.644026 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f3a0d1-4d8c-4361-b53d-32a42d98650a None None] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:40:45.645294 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f3a0d1-4d8c-4361-b53d-32a42d98650a None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:40:45.649689 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f3a0d1-4d8c-4361-b53d-32a42d98650a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:40:45.664656 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-b7052bd2-4aab-42f6-8d18-6496fa992ae5 None None] MFA Rules not processed for user `e471b31763074167ba81c2c25f39b857`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:40:45.675396 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-b7052bd2-4aab-42f6-8d18-6496fa992ae5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:40:45.705013 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.auth.core [None req-08f3a0d1-4d8c-4361-b53d-32a42d98650a None None] MFA Rules not processed for user `4ed200ce29684c5f80354fa9a8638fe2`. Rule list: `[]` (Enabled: `True`). {{(pid=80312) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:40:45.709007 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-08f3a0d1-4d8c-4361-b53d-32a42d98650a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:40:45.727060 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 911/1811] 10.222.0.43 () {58 vars in 1015 bytes} [Fri Jul 1 16:40:45 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 16:40:45.727967 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f3a0d1-4d8c-4361-b53d-32a42d98650a None None] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:40:45.767010 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f3a0d1-4d8c-4361-b53d-32a42d98650a None None] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:40:45.784933 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03b8ad2-94c6-45ad-946c-fa58f9beed0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:40:45.788337 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03b8ad2-94c6-45ad-946c-fa58f9beed0f admin admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:40:45.788813 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03b8ad2-94c6-45ad-946c-fa58f9beed0f admin admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:40:45.789830 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03b8ad2-94c6-45ad-946c-fa58f9beed0f admin admin] PATH_INFO: `/v3/roles` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:40:45.791176 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03b8ad2-94c6-45ad-946c-fa58f9beed0f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:40:45.793875 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03b8ad2-94c6-45ad-946c-fa58f9beed0f admin admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:40:45.839234 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 912/1812] 10.222.0.57 () {58 vars in 1121 bytes} [Fri Jul 1 16:40:45 2022] GET /identity/v3/roles => generated 1420 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 16:40:45.841652 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03b8ad2-94c6-45ad-946c-fa58f9beed0f admin admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:40:45.841780 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:40:45.849139 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-e03b8ad2-94c6-45ad-946c-fa58f9beed0f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:40:45.885878 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03b8ad2-94c6-45ad-946c-fa58f9beed0f admin admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:40:45.897974 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65bd0e3-eded-4ffa-aae2-3f04e41a126d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ed200ce29684c5f80354fa9a8638fe2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0788e6725a84e57a4174284c1845755', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:40:45.901121 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65bd0e3-eded-4ffa-aae2-3f04e41a126d service neutron] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:40:45.901739 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65bd0e3-eded-4ffa-aae2-3f04e41a126d service neutron] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:40:45.902168 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65bd0e3-eded-4ffa-aae2-3f04e41a126d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:40:45.913264 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:40:45.916164 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 901/1813] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 1 16:40:45 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 357 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 16:40:45.918548 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7052bd2-4aab-42f6-8d18-6496fa992ae5 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:40:45.928381 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65bd0e3-eded-4ffa-aae2-3f04e41a126d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:40:45.937554 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65bd0e3-eded-4ffa-aae2-3f04e41a126d service neutron] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:40:45.952054 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7052bd2-4aab-42f6-8d18-6496fa992ae5 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:40:45.965895 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0aa7dea-4390-4dd0-abe9-bdaa34f97c16 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ed200ce29684c5f80354fa9a8638fe2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0788e6725a84e57a4174284c1845755', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:40:45.966981 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0aa7dea-4390-4dd0-abe9-bdaa34f97c16 service neutron] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:40:45.967476 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0aa7dea-4390-4dd0-abe9-bdaa34f97c16 service neutron] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:40:45.967971 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0aa7dea-4390-4dd0-abe9-bdaa34f97c16 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:40:45.974647 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 913/1814] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 1 16:40:45 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 16:40:45.975748 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65bd0e3-eded-4ffa-aae2-3f04e41a126d service neutron] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:40:45.990689 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0aa7dea-4390-4dd0-abe9-bdaa34f97c16 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:40:45.992574 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0aa7dea-4390-4dd0-abe9-bdaa34f97c16 service neutron] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:40:46.018230 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 902/1815] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 1 16:40:45 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 16:40:46.032945 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65bd0e3-eded-4ffa-aae2-3f04e41a126d service neutron] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:40:46.059666 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2329ee8d-9d11-4e7d-a870-5d066ca288f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:40:46.060713 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2329ee8d-9d11-4e7d-a870-5d066ca288f8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:40:46.066972 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2329ee8d-9d11-4e7d-a870-5d066ca288f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:40:46.069798 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2329ee8d-9d11-4e7d-a870-5d066ca288f8 admin admin] PATH_INFO: `/v3/projects/5f3a9ddf1ad34fc489e50c0e6b0165c9/users/bc2ecf1d350147a4a3ca62be9ba78989/roles/84ee739146934c1ab32c501ff8093644` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:40:46.163457 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2329ee8d-9d11-4e7d-a870-5d066ca288f8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5f3a9ddf1ad34fc489e50c0e6b0165c9, user_id=bc2ecf1d350147a4a3ca62be9ba78989, role_id=84ee739146934c1ab32c501ff8093644)` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:40:46.185034 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2329ee8d-9d11-4e7d-a870-5d066ca288f8 admin admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:40:46.231735 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 914/1816] 10.222.0.57 () {60 vars in 1482 bytes} [Fri Jul 1 16:40:45 2022] PUT /identity/v3/projects/5f3a9ddf1ad34fc489e50c0e6b0165c9/users/bc2ecf1d350147a4a3ca62be9ba78989/roles/84ee739146934c1ab32c501ff8093644 => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 16:40:46.235862 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0aa7dea-4390-4dd0-abe9-bdaa34f97c16 service neutron] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:40:46.261805 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0aa7dea-4390-4dd0-abe9-bdaa34f97c16 service neutron] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:40:46.444241 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f29913-9306-4d41-9f9d-0aff5a4ceb2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:40:46.445317 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f29913-9306-4d41-9f9d-0aff5a4ceb2b admin admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:40:46.445641 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f29913-9306-4d41-9f9d-0aff5a4ceb2b admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:40:46.445937 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f29913-9306-4d41-9f9d-0aff5a4ceb2b admin admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:40:46.446803 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f29913-9306-4d41-9f9d-0aff5a4ceb2b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:40:46.448563 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f29913-9306-4d41-9f9d-0aff5a4ceb2b admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:40:46.500563 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 903/1817] 10.222.0.57 () {58 vars in 1121 bytes} [Fri Jul 1 16:40:46 2022] GET /identity/v3/roles => generated 1420 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 16:40:46.504529 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2329ee8d-9d11-4e7d-a870-5d066ca288f8 admin admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:40:46.528234 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2329ee8d-9d11-4e7d-a870-5d066ca288f8 admin admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:40:46.544503 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c54ceac-77f6-45f1-b1ea-6e24dcdee656 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:40:46.546540 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c54ceac-77f6-45f1-b1ea-6e24dcdee656 admin admin] REQUEST_METHOD: `PUT` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:40:46.548435 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c54ceac-77f6-45f1-b1ea-6e24dcdee656 admin admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:40:46.548836 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c54ceac-77f6-45f1-b1ea-6e24dcdee656 admin admin] PATH_INFO: `/v3/projects/5f3a9ddf1ad34fc489e50c0e6b0165c9/users/bc2ecf1d350147a4a3ca62be9ba78989/roles/84ee739146934c1ab32c501ff8093644` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:40:46.557303 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c54ceac-77f6-45f1-b1ea-6e24dcdee656 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5f3a9ddf1ad34fc489e50c0e6b0165c9, user_id=bc2ecf1d350147a4a3ca62be9ba78989, role_id=84ee739146934c1ab32c501ff8093644)` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:40:46.563913 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c54ceac-77f6-45f1-b1ea-6e24dcdee656 admin admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:40:46.608736 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 915/1818] 10.222.0.57 () {60 vars in 1482 bytes} [Fri Jul 1 16:40:46 2022] PUT /identity/v3/projects/5f3a9ddf1ad34fc489e50c0e6b0165c9/users/bc2ecf1d350147a4a3ca62be9ba78989/roles/84ee739146934c1ab32c501ff8093644 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 16:40:46.621227 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64a0ea2-fdb7-4463-885a-3593f15612a4 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:40:46.621520 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64a0ea2-fdb7-4463-885a-3593f15612a4 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:40:46.621819 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64a0ea2-fdb7-4463-885a-3593f15612a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:40:46.792136 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-f64a0ea2-fdb7-4463-885a-3593f15612a4 None None] MFA Rules not processed for user `bc2ecf1d350147a4a3ca62be9ba78989`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:40:46.793454 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-f64a0ea2-fdb7-4463-885a-3593f15612a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:40:47.107898 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 904/1819] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 1 16:40:46 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 488 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 16:40:47.273774 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c54ceac-77f6-45f1-b1ea-6e24dcdee656 admin admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:40:47.504525 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c54ceac-77f6-45f1-b1ea-6e24dcdee656 admin admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:40:47.524173 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92655ee8-3971-4d5e-9fa1-77527043e535 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbddf971a2354bbdbb9da1d63f3abf88', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0788e6725a84e57a4174284c1845755', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:40:47.524886 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92655ee8-3971-4d5e-9fa1-77527043e535 service cinder] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:40:47.525239 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92655ee8-3971-4d5e-9fa1-77527043e535 service cinder] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:40:47.530018 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92655ee8-3971-4d5e-9fa1-77527043e535 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:40:47.575281 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92655ee8-3971-4d5e-9fa1-77527043e535 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:40:47.581977 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92655ee8-3971-4d5e-9fa1-77527043e535 service cinder] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:40:47.644753 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 916/1820] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 1 16:40:47 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 01 16:40:48.758899 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64a0ea2-fdb7-4463-885a-3593f15612a4 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:40:48.761057 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-f64a0ea2-fdb7-4463-885a-3593f15612a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:40:49.084727 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:40:49.128830 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64a0ea2-fdb7-4463-885a-3593f15612a4 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:40:49.147055 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55a78dc-27b6-458d-8939-f048dfe1ef01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:40:49.148241 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55a78dc-27b6-458d-8939-f048dfe1ef01 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:40:49.148730 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55a78dc-27b6-458d-8939-f048dfe1ef01 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:40:49.149190 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55a78dc-27b6-458d-8939-f048dfe1ef01 admin admin] PATH_INFO: `/v3/users/00ab33726a1b4cfda4b49a0bb4f88348` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:40:49.150884 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f55a78dc-27b6-458d-8939-f048dfe1ef01 admin admin] RBAC: Authorizing `identity:delete_user(user_id=00ab33726a1b4cfda4b49a0bb4f88348)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:40:49.159762 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f55a78dc-27b6-458d-8939-f048dfe1ef01 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:40:49.297087 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-f55a78dc-27b6-458d-8939-f048dfe1ef01 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '00ab33726a1b4cfda4b49a0bb4f88348'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:40:49.300845 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-f55a78dc-27b6-458d-8939-f048dfe1ef01 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '00ab33726a1b4cfda4b49a0bb4f88348'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:40:49.302867 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-f55a78dc-27b6-458d-8939-f048dfe1ef01 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '00ab33726a1b4cfda4b49a0bb4f88348'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:40:49.318614 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-f55a78dc-27b6-458d-8939-f048dfe1ef01 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '00ab33726a1b4cfda4b49a0bb4f88348'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:40:49.376762 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-f55a78dc-27b6-458d-8939-f048dfe1ef01 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '00ab33726a1b4cfda4b49a0bb4f88348'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:40:49.408505 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 905/1821] 10.222.0.57 () {58 vars in 1223 bytes} [Fri Jul 1 16:40:48 2022] DELETE /identity/v3/users/00ab33726a1b4cfda4b49a0bb4f88348 => generated 0 bytes in 650 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 16:40:49.414365 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee59de3-b121-4a91-b2f3-8cc71af5c834 None None] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:40:49.414957 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee59de3-b121-4a91-b2f3-8cc71af5c834 None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:40:49.415150 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee59de3-b121-4a91-b2f3-8cc71af5c834 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:40:49.490677 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.auth.core [None req-8ee59de3-b121-4a91-b2f3-8cc71af5c834 None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80312) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:40:49.496554 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-8ee59de3-b121-4a91-b2f3-8cc71af5c834 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:40:49.736062 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 917/1822] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 1 16:40:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 323 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 16:40:49.793781 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55a78dc-27b6-458d-8939-f048dfe1ef01 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:40:49.802444 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-f55a78dc-27b6-458d-8939-f048dfe1ef01 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:40:49.964763 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55a78dc-27b6-458d-8939-f048dfe1ef01 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:40:49.969625 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef9e7a1-24b2-4887-8eb6-a383a60d0dd3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ed200ce29684c5f80354fa9a8638fe2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0788e6725a84e57a4174284c1845755', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:40:49.970614 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef9e7a1-24b2-4887-8eb6-a383a60d0dd3 service neutron] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:40:49.971591 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef9e7a1-24b2-4887-8eb6-a383a60d0dd3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:40:49.972006 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef9e7a1-24b2-4887-8eb6-a383a60d0dd3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:40:49.994441 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef9e7a1-24b2-4887-8eb6-a383a60d0dd3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:40:49.996007 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef9e7a1-24b2-4887-8eb6-a383a60d0dd3 service neutron] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:40:50.030611 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 906/1823] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 1 16:40:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 237 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 16:40:50.077204 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee59de3-b121-4a91-b2f3-8cc71af5c834 None None] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:40:50.079604 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-8ee59de3-b121-4a91-b2f3-8cc71af5c834 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:40:50.101497 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee59de3-b121-4a91-b2f3-8cc71af5c834 None None] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:40:50.110833 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05dac49b-7f76-4829-8e18-d22f2da0bcaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:40:50.112114 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05dac49b-7f76-4829-8e18-d22f2da0bcaa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:40:50.112602 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05dac49b-7f76-4829-8e18-d22f2da0bcaa admin admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:40:50.113127 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05dac49b-7f76-4829-8e18-d22f2da0bcaa admin admin] PATH_INFO: `/v3/users/bc263c83b4ac49a2a81f057c1ab031e4` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:40:50.114747 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05dac49b-7f76-4829-8e18-d22f2da0bcaa admin admin] RBAC: Authorizing `identity:delete_user(user_id=bc263c83b4ac49a2a81f057c1ab031e4)` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:40:50.121690 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05dac49b-7f76-4829-8e18-d22f2da0bcaa admin admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:40:50.235937 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-05dac49b-7f76-4829-8e18-d22f2da0bcaa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bc263c83b4ac49a2a81f057c1ab031e4'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:40:50.245144 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=90,keystone:INSERT=2,keystone:DELETE=8 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:40:50.305227 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-05dac49b-7f76-4829-8e18-d22f2da0bcaa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bc263c83b4ac49a2a81f057c1ab031e4'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:40:50.389037 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-05dac49b-7f76-4829-8e18-d22f2da0bcaa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bc263c83b4ac49a2a81f057c1ab031e4'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:40:50.400992 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-05dac49b-7f76-4829-8e18-d22f2da0bcaa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bc263c83b4ac49a2a81f057c1ab031e4'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:40:50.405264 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-05dac49b-7f76-4829-8e18-d22f2da0bcaa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bc263c83b4ac49a2a81f057c1ab031e4'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:40:50.454298 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7489fb2b-33de-4f88-84e2-7489aa16b7ae None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:40:50.455042 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7489fb2b-33de-4f88-84e2-7489aa16b7ae None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:40:50.458049 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7489fb2b-33de-4f88-84e2-7489aa16b7ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:40:50.472856 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 918/1824] 10.222.0.57 () {58 vars in 1223 bytes} [Fri Jul 1 16:40:50 2022] DELETE /identity/v3/users/bc263c83b4ac49a2a81f057c1ab031e4 => generated 0 bytes in 396 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 16:40:50.480563 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cfe2f1-3985-48d7-aa68-96d38c8f4f48 None None] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:40:50.482038 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cfe2f1-3985-48d7-aa68-96d38c8f4f48 None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:40:50.485038 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cfe2f1-3985-48d7-aa68-96d38c8f4f48 None None] PATH_INFO: `/` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:40:50.491548 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 919/1825] 10.222.0.57 () {54 vars in 921 bytes} [Fri Jul 1 16:40:50 2022] GET /identity => generated 270 bytes in 13 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 16:40:50.496250 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447a7757-95cc-48a9-85e6-64b179cdd510 None None] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:40:50.501095 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447a7757-95cc-48a9-85e6-64b179cdd510 None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:40:50.501787 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447a7757-95cc-48a9-85e6-64b179cdd510 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:40:50.512636 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-7489fb2b-33de-4f88-84e2-7489aa16b7ae None None] MFA Rules not processed for user `71c383c04f9543d8ab97a6c0db268ca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:40:50.516820 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-7489fb2b-33de-4f88-84e2-7489aa16b7ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:40:50.551886 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.auth.core [None req-447a7757-95cc-48a9-85e6-64b179cdd510 None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80312) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:40:50.553229 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-447a7757-95cc-48a9-85e6-64b179cdd510 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:40:50.671762 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 907/1826] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 1 16:40:50 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 16:40:50.674989 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bfae7e-0b0b-49b9-9154-428009c5c4ce None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:40:50.677751 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bfae7e-0b0b-49b9-9154-428009c5c4ce None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:40:50.677751 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bfae7e-0b0b-49b9-9154-428009c5c4ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:40:50.763205 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-e5bfae7e-0b0b-49b9-9154-428009c5c4ce None None] MFA Rules not processed for user `7a50f745b31941b7a7265c0fc852f48e`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:40:50.768056 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-e5bfae7e-0b0b-49b9-9154-428009c5c4ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:40:50.831819 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 920/1827] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 1 16:40:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 337 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 16:40:50.837059 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:40:50.881656 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447a7757-95cc-48a9-85e6-64b179cdd510 None None] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:40:50.884258 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-447a7757-95cc-48a9-85e6-64b179cdd510 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:40:50.998685 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 908/1828] 10.222.0.57 () {58 vars in 1018 bytes} [Fri Jul 1 16:40:50 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 325 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 16:40:51.025819 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447a7757-95cc-48a9-85e6-64b179cdd510 None None] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:40:51.035170 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b40da4f-bbd7-4992-b74c-d8ad9ad37bbc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ed200ce29684c5f80354fa9a8638fe2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0788e6725a84e57a4174284c1845755', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:40:51.035734 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b40da4f-bbd7-4992-b74c-d8ad9ad37bbc service neutron] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:40:51.036023 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b40da4f-bbd7-4992-b74c-d8ad9ad37bbc service neutron] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:40:51.036304 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b40da4f-bbd7-4992-b74c-d8ad9ad37bbc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:40:51.067651 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b40da4f-bbd7-4992-b74c-d8ad9ad37bbc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:40:51.071962 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b40da4f-bbd7-4992-b74c-d8ad9ad37bbc service neutron] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:40:51.081152 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5bfae7e-0b0b-49b9-9154-428009c5c4ce None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:40:51.082754 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-e5bfae7e-0b0b-49b9-9154-428009c5c4ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:40:51.095030 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 921/1829] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 1 16:40:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 214 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 16:40:51.112508 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5bfae7e-0b0b-49b9-9154-428009c5c4ce None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:40:51.120716 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e9a8f2-7394-4a60-a76c-496453fc35d3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a50f745b31941b7a7265c0fc852f48e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0788e6725a84e57a4174284c1845755', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:40:51.122468 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e9a8f2-7394-4a60-a76c-496453fc35d3 service nova] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:40:51.122984 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e9a8f2-7394-4a60-a76c-496453fc35d3 service nova] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:40:51.123486 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e9a8f2-7394-4a60-a76c-496453fc35d3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:40:51.135196 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e9a8f2-7394-4a60-a76c-496453fc35d3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:40:51.137088 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e9a8f2-7394-4a60-a76c-496453fc35d3 service nova] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:40:51.160565 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 909/1830] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 1 16:40:51 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 80 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 01 16:40:51.765699 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b40da4f-bbd7-4992-b74c-d8ad9ad37bbc service neutron] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:40:51.768365 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-7b40da4f-bbd7-4992-b74c-d8ad9ad37bbc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:40:51.793061 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b40da4f-bbd7-4992-b74c-d8ad9ad37bbc service neutron] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:40:51.806560 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9798b6-1a0d-4ab4-ac0f-34c61ea26489 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:40:51.808656 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9798b6-1a0d-4ab4-ac0f-34c61ea26489 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:40:51.809167 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9798b6-1a0d-4ab4-ac0f-34c61ea26489 admin admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:40:51.809648 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9798b6-1a0d-4ab4-ac0f-34c61ea26489 admin admin] PATH_INFO: `/v3/projects/37c1044c791040508147811f544e2665` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:40:51.812493 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9798b6-1a0d-4ab4-ac0f-34c61ea26489 admin admin] RBAC: Authorizing `identity:delete_project(project_id=37c1044c791040508147811f544e2665)` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:40:51.818108 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9798b6-1a0d-4ab4-ac0f-34c61ea26489 admin admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:40:52.002148 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e9a8f2-7394-4a60-a76c-496453fc35d3 service nova] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:40:52.005425 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-06e9a8f2-7394-4a60-a76c-496453fc35d3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:40:52.020786 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-9f9798b6-1a0d-4ab4-ac0f-34c61ea26489 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '37c1044c791040508147811f544e2665', 'request_id': 'req-9f9798b6-1a0d-4ab4-ac0f-34c61ea26489'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:40:52.051413 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-9f9798b6-1a0d-4ab4-ac0f-34c61ea26489 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '37c1044c791040508147811f544e2665', 'request_id': 'req-9f9798b6-1a0d-4ab4-ac0f-34c61ea26489'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:40:52.062329 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e9a8f2-7394-4a60-a76c-496453fc35d3 service nova] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:40:52.082940 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-9f9798b6-1a0d-4ab4-ac0f-34c61ea26489 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '37c1044c791040508147811f544e2665', 'request_id': 'req-9f9798b6-1a0d-4ab4-ac0f-34c61ea26489'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:40:52.109735 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=89,keystone:DELETE=9,keystone:INSERT=2 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:40:52.132519 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-9f9798b6-1a0d-4ab4-ac0f-34c61ea26489 admin admin] The token cache is being invalidate because project 37c1044c791040508147811f544e2665 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80312) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 01 16:40:52.135816 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-9f9798b6-1a0d-4ab4-ac0f-34c61ea26489 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:40:52.140513 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-9f9798b6-1a0d-4ab4-ac0f-34c61ea26489 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:40:52.147896 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 922/1831] 10.222.0.57 () {58 vars in 1232 bytes} [Fri Jul 1 16:40:51 2022] DELETE /identity/v3/projects/37c1044c791040508147811f544e2665 => generated 0 bytes in 381 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 16:40:52.153619 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9798b6-1a0d-4ab4-ac0f-34c61ea26489 admin admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:40:52.156612 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-9f9798b6-1a0d-4ab4-ac0f-34c61ea26489 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:40:52.254284 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45edac4-3c0b-4d95-8c56-072d1536919e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:40:52.255917 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45edac4-3c0b-4d95-8c56-072d1536919e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:40:52.256418 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45edac4-3c0b-4d95-8c56-072d1536919e admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:40:52.256953 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45edac4-3c0b-4d95-8c56-072d1536919e admin admin] PATH_INFO: `/v3/projects/725874a440d444e59ed18043b614dbee` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:40:52.258666 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f45edac4-3c0b-4d95-8c56-072d1536919e admin admin] RBAC: Authorizing `identity:delete_project(project_id=725874a440d444e59ed18043b614dbee)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:40:52.262231 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f45edac4-3c0b-4d95-8c56-072d1536919e admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:40:52.356282 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9798b6-1a0d-4ab4-ac0f-34c61ea26489 admin admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:40:52.423454 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-f45edac4-3c0b-4d95-8c56-072d1536919e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '725874a440d444e59ed18043b614dbee', 'request_id': 'req-f45edac4-3c0b-4d95-8c56-072d1536919e'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:40:52.460928 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-f45edac4-3c0b-4d95-8c56-072d1536919e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '725874a440d444e59ed18043b614dbee', 'request_id': 'req-f45edac4-3c0b-4d95-8c56-072d1536919e'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:40:52.490676 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-f45edac4-3c0b-4d95-8c56-072d1536919e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '725874a440d444e59ed18043b614dbee', 'request_id': 'req-f45edac4-3c0b-4d95-8c56-072d1536919e'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:40:52.516714 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f41e73d-8254-4c77-8718-2225660cbc6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:40:52.518432 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f41e73d-8254-4c77-8718-2225660cbc6d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:40:52.519037 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f41e73d-8254-4c77-8718-2225660cbc6d admin admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:40:52.522216 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-f45edac4-3c0b-4d95-8c56-072d1536919e admin admin] The token cache is being invalidate because project 725874a440d444e59ed18043b614dbee was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80311) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 01 16:40:52.522566 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-f45edac4-3c0b-4d95-8c56-072d1536919e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:40:52.523226 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f41e73d-8254-4c77-8718-2225660cbc6d admin admin] PATH_INFO: `/v3/users/16624875454e4a39a86aeeebcd1bb874` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:40:52.523329 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-f45edac4-3c0b-4d95-8c56-072d1536919e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:40:52.525764 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 910/1832] 10.222.0.57 () {58 vars in 1232 bytes} [Fri Jul 1 16:40:52 2022] DELETE /identity/v3/projects/725874a440d444e59ed18043b614dbee => generated 0 bytes in 524 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 16:40:52.526259 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f41e73d-8254-4c77-8718-2225660cbc6d admin admin] RBAC: Authorizing `identity:delete_user(user_id=16624875454e4a39a86aeeebcd1bb874)` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:40:52.530812 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f41e73d-8254-4c77-8718-2225660cbc6d admin admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:40:52.530955 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45edac4-3c0b-4d95-8c56-072d1536919e admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:40:52.545418 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-f45edac4-3c0b-4d95-8c56-072d1536919e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:40:52.565966 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45edac4-3c0b-4d95-8c56-072d1536919e admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:40:52.571595 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6491b401-fc86-49e3-a360-42fb3ff7be4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:40:52.572655 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6491b401-fc86-49e3-a360-42fb3ff7be4d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:40:52.573050 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6491b401-fc86-49e3-a360-42fb3ff7be4d admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:40:52.573438 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6491b401-fc86-49e3-a360-42fb3ff7be4d admin admin] PATH_INFO: `/v3/users/bc2ecf1d350147a4a3ca62be9ba78989` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:40:52.574892 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6491b401-fc86-49e3-a360-42fb3ff7be4d admin admin] RBAC: Authorizing `identity:delete_user(user_id=bc2ecf1d350147a4a3ca62be9ba78989)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:40:52.579060 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6491b401-fc86-49e3-a360-42fb3ff7be4d admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:40:52.656513 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:40:52.707795 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-4f41e73d-8254-4c77-8718-2225660cbc6d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '16624875454e4a39a86aeeebcd1bb874'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:40:52.733555 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-6491b401-fc86-49e3-a360-42fb3ff7be4d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bc2ecf1d350147a4a3ca62be9ba78989'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:40:52.735344 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-6491b401-fc86-49e3-a360-42fb3ff7be4d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bc2ecf1d350147a4a3ca62be9ba78989'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:40:52.736184 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-6491b401-fc86-49e3-a360-42fb3ff7be4d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bc2ecf1d350147a4a3ca62be9ba78989'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:40:52.740823 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-4f41e73d-8254-4c77-8718-2225660cbc6d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '16624875454e4a39a86aeeebcd1bb874'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:40:52.752369 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-6491b401-fc86-49e3-a360-42fb3ff7be4d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bc2ecf1d350147a4a3ca62be9ba78989'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:40:52.770303 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-4f41e73d-8254-4c77-8718-2225660cbc6d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '16624875454e4a39a86aeeebcd1bb874'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:40:52.778231 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-4f41e73d-8254-4c77-8718-2225660cbc6d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '16624875454e4a39a86aeeebcd1bb874'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:40:52.779505 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-4f41e73d-8254-4c77-8718-2225660cbc6d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '16624875454e4a39a86aeeebcd1bb874'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:40:52.806680 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-6491b401-fc86-49e3-a360-42fb3ff7be4d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bc2ecf1d350147a4a3ca62be9ba78989'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:40:52.812201 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 923/1833] 10.222.0.57 () {58 vars in 1223 bytes} [Fri Jul 1 16:40:52 2022] DELETE /identity/v3/users/16624875454e4a39a86aeeebcd1bb874 => generated 0 bytes in 661 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 16:40:52.858669 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 911/1834] 10.222.0.57 () {58 vars in 1223 bytes} [Fri Jul 1 16:40:52 2022] DELETE /identity/v3/users/bc2ecf1d350147a4a3ca62be9ba78989 => generated 0 bytes in 328 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 16:40:54.646937 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b29838-8757-443d-b7dc-9f364221af01 None None] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:40:54.655409 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6491b401-fc86-49e3-a360-42fb3ff7be4d admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:40:54.655911 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b29838-8757-443d-b7dc-9f364221af01 None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:40:54.657228 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b29838-8757-443d-b7dc-9f364221af01 None None] PATH_INFO: `/` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:40:54.659437 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-6491b401-fc86-49e3-a360-42fb3ff7be4d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:40:54.663674 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 924/1835] 10.222.0.57 () {54 vars in 921 bytes} [Fri Jul 1 16:40:54 2022] GET /identity => generated 270 bytes in 18 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 16:40:54.686934 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40580f3f-8b88-443a-aeed-175f49213fc3 None None] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:40:54.687258 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40580f3f-8b88-443a-aeed-175f49213fc3 None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:40:54.687504 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40580f3f-8b88-443a-aeed-175f49213fc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:40:54.834061 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6491b401-fc86-49e3-a360-42fb3ff7be4d admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:40:54.848151 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c514d70a-ec95-4101-9ef6-8a32e9cbd7b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:40:54.849592 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c514d70a-ec95-4101-9ef6-8a32e9cbd7b8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:40:54.850610 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c514d70a-ec95-4101-9ef6-8a32e9cbd7b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:40:54.851229 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c514d70a-ec95-4101-9ef6-8a32e9cbd7b8 admin admin] PATH_INFO: `/v3/projects/52a7aaae39084cdf84c1044da9627eda` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:40:54.853032 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c514d70a-ec95-4101-9ef6-8a32e9cbd7b8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=52a7aaae39084cdf84c1044da9627eda)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:40:54.858497 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c514d70a-ec95-4101-9ef6-8a32e9cbd7b8 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:40:54.903337 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.auth.core [None req-40580f3f-8b88-443a-aeed-175f49213fc3 None None] MFA Rules not processed for user `7a50f745b31941b7a7265c0fc852f48e`. Rule list: `[]` (Enabled: `True`). {{(pid=80312) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:40:54.908220 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-40580f3f-8b88-443a-aeed-175f49213fc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:40:55.015053 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:40:55.016572 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-c514d70a-ec95-4101-9ef6-8a32e9cbd7b8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '52a7aaae39084cdf84c1044da9627eda', 'request_id': 'req-c514d70a-ec95-4101-9ef6-8a32e9cbd7b8'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:40:55.028905 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-c514d70a-ec95-4101-9ef6-8a32e9cbd7b8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '52a7aaae39084cdf84c1044da9627eda', 'request_id': 'req-c514d70a-ec95-4101-9ef6-8a32e9cbd7b8'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:40:55.043623 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-c514d70a-ec95-4101-9ef6-8a32e9cbd7b8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '52a7aaae39084cdf84c1044da9627eda', 'request_id': 'req-c514d70a-ec95-4101-9ef6-8a32e9cbd7b8'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:40:55.070906 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-c514d70a-ec95-4101-9ef6-8a32e9cbd7b8 admin admin] The token cache is being invalidate because project 52a7aaae39084cdf84c1044da9627eda was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80311) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 01 16:40:55.071721 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-c514d70a-ec95-4101-9ef6-8a32e9cbd7b8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:40:55.072965 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-c514d70a-ec95-4101-9ef6-8a32e9cbd7b8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:40:55.082042 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 912/1836] 10.222.0.57 () {58 vars in 1232 bytes} [Fri Jul 1 16:40:54 2022] DELETE /identity/v3/projects/52a7aaae39084cdf84c1044da9627eda => generated 0 bytes in 422 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 16:40:55.082042 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c514d70a-ec95-4101-9ef6-8a32e9cbd7b8 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:40:55.084439 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-c514d70a-ec95-4101-9ef6-8a32e9cbd7b8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:40:55.153672 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 925/1837] 10.222.0.57 () {58 vars in 1018 bytes} [Fri Jul 1 16:40:54 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 467 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 01 16:40:55.155407 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f938c5a-2fa6-485f-af95-d38f4d5b355e None None] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:40:55.155702 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f938c5a-2fa6-485f-af95-d38f4d5b355e None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:40:55.156221 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f938c5a-2fa6-485f-af95-d38f4d5b355e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:40:55.192000 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:DELETE=14,keystone:SELECT=84,keystone:INSERT=2 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:40:55.206294 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.auth.core [None req-4f938c5a-2fa6-485f-af95-d38f4d5b355e None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80312) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:40:55.213817 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-4f938c5a-2fa6-485f-af95-d38f4d5b355e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:40:55.290130 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c514d70a-ec95-4101-9ef6-8a32e9cbd7b8 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:40:55.311183 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325b2d4b-d09b-4e94-ad0c-24b792ad6b47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:40:55.312180 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325b2d4b-d09b-4e94-ad0c-24b792ad6b47 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:40:55.312636 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325b2d4b-d09b-4e94-ad0c-24b792ad6b47 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:40:55.313003 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325b2d4b-d09b-4e94-ad0c-24b792ad6b47 admin admin] PATH_INFO: `/v3/projects/5f3a9ddf1ad34fc489e50c0e6b0165c9` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:40:55.314443 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-325b2d4b-d09b-4e94-ad0c-24b792ad6b47 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5f3a9ddf1ad34fc489e50c0e6b0165c9)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:40:55.316513 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-325b2d4b-d09b-4e94-ad0c-24b792ad6b47 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:40:55.413539 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 926/1838] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 1 16:40:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 251 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 16:40:55.413539 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f938c5a-2fa6-485f-af95-d38f4d5b355e None None] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:40:55.417285 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-4f938c5a-2fa6-485f-af95-d38f4d5b355e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:40:55.534212 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-325b2d4b-d09b-4e94-ad0c-24b792ad6b47 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5f3a9ddf1ad34fc489e50c0e6b0165c9', 'request_id': 'req-325b2d4b-d09b-4e94-ad0c-24b792ad6b47'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:40:55.557971 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-325b2d4b-d09b-4e94-ad0c-24b792ad6b47 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5f3a9ddf1ad34fc489e50c0e6b0165c9', 'request_id': 'req-325b2d4b-d09b-4e94-ad0c-24b792ad6b47'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:40:55.569528 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-325b2d4b-d09b-4e94-ad0c-24b792ad6b47 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5f3a9ddf1ad34fc489e50c0e6b0165c9', 'request_id': 'req-325b2d4b-d09b-4e94-ad0c-24b792ad6b47'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:40:55.601087 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-325b2d4b-d09b-4e94-ad0c-24b792ad6b47 admin admin] The token cache is being invalidate because project 5f3a9ddf1ad34fc489e50c0e6b0165c9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80311) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 01 16:40:55.601540 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-325b2d4b-d09b-4e94-ad0c-24b792ad6b47 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:40:55.603463 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-325b2d4b-d09b-4e94-ad0c-24b792ad6b47 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:40:55.606107 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 913/1839] 10.222.0.57 () {58 vars in 1232 bytes} [Fri Jul 1 16:40:55 2022] DELETE /identity/v3/projects/5f3a9ddf1ad34fc489e50c0e6b0165c9 => generated 0 bytes in 528 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 16:40:55.608461 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ea1306-c872-4ee6-b391-2ee0157cd6e3 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:40:55.608887 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ea1306-c872-4ee6-b391-2ee0157cd6e3 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:40:55.609076 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ea1306-c872-4ee6-b391-2ee0157cd6e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:40:55.638811 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-26ea1306-c872-4ee6-b391-2ee0157cd6e3 None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:40:55.640527 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-26ea1306-c872-4ee6-b391-2ee0157cd6e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:40:55.654285 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f938c5a-2fa6-485f-af95-d38f4d5b355e None None] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:40:55.744654 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=100 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:40:55.774747 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 914/1840] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 1 16:40:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 16:40:55.776219 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c73e3ef-64d7-4307-be48-9d28cb1bbbdf None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:40:55.776512 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c73e3ef-64d7-4307-be48-9d28cb1bbbdf None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:40:55.776793 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c73e3ef-64d7-4307-be48-9d28cb1bbbdf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:40:55.777613 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a36553-5edf-496b-813e-e7700ed87992 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a50f745b31941b7a7265c0fc852f48e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0788e6725a84e57a4174284c1845755', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:40:55.778956 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a36553-5edf-496b-813e-e7700ed87992 service nova] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:40:55.779493 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a36553-5edf-496b-813e-e7700ed87992 service nova] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:40:55.779882 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a36553-5edf-496b-813e-e7700ed87992 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:40:55.784078 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-24a36553-5edf-496b-813e-e7700ed87992 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:40:55.808043 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-8c73e3ef-64d7-4307-be48-9d28cb1bbbdf None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:40:55.809702 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-8c73e3ef-64d7-4307-be48-9d28cb1bbbdf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:40:55.816161 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a36553-5edf-496b-813e-e7700ed87992 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:40:55.817648 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a36553-5edf-496b-813e-e7700ed87992 service nova] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:40:55.825019 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 915/1841] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 1 16:40:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 16:40:55.825984 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c73e3ef-64d7-4307-be48-9d28cb1bbbdf None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:40:55.837788 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c73e3ef-64d7-4307-be48-9d28cb1bbbdf None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:40:55.847825 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac66e65a-1d33-4ac3-aa18-4853c7e79332 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:40:55.847825 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac66e65a-1d33-4ac3-aa18-4853c7e79332 admin admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:40:55.847825 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac66e65a-1d33-4ac3-aa18-4853c7e79332 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:40:55.847825 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac66e65a-1d33-4ac3-aa18-4853c7e79332 admin admin] PATH_INFO: `/v3/domains` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:40:55.848141 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac66e65a-1d33-4ac3-aa18-4853c7e79332 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:40:55.848141 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac66e65a-1d33-4ac3-aa18-4853c7e79332 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:40:55.848141 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac66e65a-1d33-4ac3-aa18-4853c7e79332 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:40:55.851452 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 927/1842] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 1 16:40:55 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 445 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 16:40:55.852979 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88b99af-fc6f-4b5f-a751-f92c4c3aaf2c None None] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:40:55.853276 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88b99af-fc6f-4b5f-a751-f92c4c3aaf2c None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:40:55.853610 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88b99af-fc6f-4b5f-a751-f92c4c3aaf2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:40:55.863215 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 916/1843] 10.222.0.57 () {58 vars in 1152 bytes} [Fri Jul 1 16:40:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:40:55.871359 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac66e65a-1d33-4ac3-aa18-4853c7e79332 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:40:55.903290 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac66e65a-1d33-4ac3-aa18-4853c7e79332 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:40:55.908172 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3088a7-57ac-4b8c-bb15-537c836ba5d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:40:55.909576 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3088a7-57ac-4b8c-bb15-537c836ba5d4 admin admin] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:40:55.909942 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3088a7-57ac-4b8c-bb15-537c836ba5d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:40:55.910378 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3088a7-57ac-4b8c-bb15-537c836ba5d4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:40:55.911027 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c3088a7-57ac-4b8c-bb15-537c836ba5d4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:40:55.912195 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.auth.core [None req-e88b99af-fc6f-4b5f-a751-f92c4c3aaf2c None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80312) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:40:55.913297 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c3088a7-57ac-4b8c-bb15-537c836ba5d4 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:40:55.914286 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-e88b99af-fc6f-4b5f-a751-f92c4c3aaf2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:40:55.924549 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 928/1844] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 1 16:40:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 73 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 16:40:55.930012 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88b99af-fc6f-4b5f-a751-f92c4c3aaf2c None None] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:40:55.944800 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88b99af-fc6f-4b5f-a751-f92c4c3aaf2c None None] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:40:55.950799 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96e39d5-d398-49ba-96be-3d5f2e49194d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:40:55.951610 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96e39d5-d398-49ba-96be-3d5f2e49194d admin admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:40:55.952348 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96e39d5-d398-49ba-96be-3d5f2e49194d admin admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:40:55.952737 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96e39d5-d398-49ba-96be-3d5f2e49194d admin admin] PATH_INFO: `/v3/domains` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:40:55.953619 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b96e39d5-d398-49ba-96be-3d5f2e49194d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80312) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:40:55.954029 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b96e39d5-d398-49ba-96be-3d5f2e49194d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:40:55.957058 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b96e39d5-d398-49ba-96be-3d5f2e49194d admin admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:40:55.959789 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 917/1845] 10.222.0.57 () {60 vars in 1152 bytes} [Fri Jul 1 16:40:55 2022] POST /identity/v3/projects => generated 369 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 16:40:55.963633 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3088a7-57ac-4b8c-bb15-537c836ba5d4 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:40:55.978706 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 929/1846] 10.222.0.57 () {58 vars in 1152 bytes} [Fri Jul 1 16:40:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:40:55.980527 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96e39d5-d398-49ba-96be-3d5f2e49194d admin admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:40:55.993460 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96e39d5-d398-49ba-96be-3d5f2e49194d admin admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:40:55.997046 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:40:56.062927 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3088a7-57ac-4b8c-bb15-537c836ba5d4 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:40:56.070043 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4785c01-8111-47d2-8dd1-396545071716 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:40:56.071049 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4785c01-8111-47d2-8dd1-396545071716 admin admin] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:40:56.071675 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4785c01-8111-47d2-8dd1-396545071716 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:40:56.072019 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4785c01-8111-47d2-8dd1-396545071716 admin admin] PATH_INFO: `/v3/users` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:40:56.072789 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4785c01-8111-47d2-8dd1-396545071716 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:40:56.074785 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4785c01-8111-47d2-8dd1-396545071716 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:40:56.081840 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b24ea0-ef1c-4858-be0e-51fbc9faf584 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:40:56.082529 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b24ea0-ef1c-4858-be0e-51fbc9faf584 admin admin] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:40:56.082821 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b24ea0-ef1c-4858-be0e-51fbc9faf584 admin admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:40:56.083103 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b24ea0-ef1c-4858-be0e-51fbc9faf584 admin admin] PATH_INFO: `/v3/projects` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:40:56.083820 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b24ea0-ef1c-4858-be0e-51fbc9faf584 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:40:56.086328 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b24ea0-ef1c-4858-be0e-51fbc9faf584 admin admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:40:56.103757 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 918/1847] 10.222.0.57 () {60 vars in 1143 bytes} [Fri Jul 1 16:40:55 2022] POST /identity/v3/users => generated 285 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 01 16:40:56.105432 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4785c01-8111-47d2-8dd1-396545071716 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:40:56.118519 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 930/1848] 10.222.0.57 () {60 vars in 1152 bytes} [Fri Jul 1 16:40:55 2022] POST /identity/v3/projects => generated 359 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 16:40:56.123800 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4785c01-8111-47d2-8dd1-396545071716 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:40:56.124917 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b24ea0-ef1c-4858-be0e-51fbc9faf584 admin admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:40:56.185587 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7f1bcf-0ca5-4c6d-ae91-12952c4c6465 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:40:56.186264 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7f1bcf-0ca5-4c6d-ae91-12952c4c6465 admin admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:40:56.186581 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7f1bcf-0ca5-4c6d-ae91-12952c4c6465 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:40:56.186866 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7f1bcf-0ca5-4c6d-ae91-12952c4c6465 admin admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:40:56.187602 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f7f1bcf-0ca5-4c6d-ae91-12952c4c6465 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:40:56.189214 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f7f1bcf-0ca5-4c6d-ae91-12952c4c6465 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:40:56.199395 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b24ea0-ef1c-4858-be0e-51fbc9faf584 admin admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:40:56.201868 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 919/1849] 10.222.0.57 () {58 vars in 1121 bytes} [Fri Jul 1 16:40:56 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 16:40:56.204717 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269627c7-ac1c-4291-9e63-3b672ba41652 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:40:56.206145 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269627c7-ac1c-4291-9e63-3b672ba41652 admin admin] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:40:56.206315 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7f1bcf-0ca5-4c6d-ae91-12952c4c6465 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:40:56.206457 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269627c7-ac1c-4291-9e63-3b672ba41652 admin admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:40:56.206457 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269627c7-ac1c-4291-9e63-3b672ba41652 admin admin] PATH_INFO: `/v3/users` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:40:56.207139 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-269627c7-ac1c-4291-9e63-3b672ba41652 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:40:56.209548 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-269627c7-ac1c-4291-9e63-3b672ba41652 admin admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:40:56.218261 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7f1bcf-0ca5-4c6d-ae91-12952c4c6465 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:40:56.224719 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a85d29-b2de-4829-b70b-b9c0d12cb3ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:40:56.225609 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a85d29-b2de-4829-b70b-b9c0d12cb3ba admin admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:40:56.225877 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a85d29-b2de-4829-b70b-b9c0d12cb3ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:40:56.226147 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a85d29-b2de-4829-b70b-b9c0d12cb3ba admin admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:40:56.227067 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a85d29-b2de-4829-b70b-b9c0d12cb3ba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:40:56.228865 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a85d29-b2de-4829-b70b-b9c0d12cb3ba admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:40:56.230680 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 931/1850] 10.222.0.57 () {60 vars in 1143 bytes} [Fri Jul 1 16:40:56 2022] POST /identity/v3/users => generated 280 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 16:40:56.235279 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269627c7-ac1c-4291-9e63-3b672ba41652 admin admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:40:56.242669 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 920/1851] 10.222.0.57 () {58 vars in 1121 bytes} [Fri Jul 1 16:40:56 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 16:40:56.246216 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269627c7-ac1c-4291-9e63-3b672ba41652 admin admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:40:56.246908 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a85d29-b2de-4829-b70b-b9c0d12cb3ba admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:40:56.250939 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22142167-b956-418a-ae26-dd77efce0316 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:40:56.251596 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22142167-b956-418a-ae26-dd77efce0316 admin admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:40:56.251880 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22142167-b956-418a-ae26-dd77efce0316 admin admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:40:56.252376 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22142167-b956-418a-ae26-dd77efce0316 admin admin] PATH_INFO: `/v3/roles` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:40:56.253670 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22142167-b956-418a-ae26-dd77efce0316 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:40:56.255304 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22142167-b956-418a-ae26-dd77efce0316 admin admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:40:56.258916 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a85d29-b2de-4829-b70b-b9c0d12cb3ba admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:40:56.264088 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b66185-ba1f-4b77-b91a-acaed9ede2ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:40:56.264971 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b66185-ba1f-4b77-b91a-acaed9ede2ca admin admin] REQUEST_METHOD: `PUT` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:40:56.265258 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b66185-ba1f-4b77-b91a-acaed9ede2ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:40:56.265541 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b66185-ba1f-4b77-b91a-acaed9ede2ca admin admin] PATH_INFO: `/v3/projects/89de13e71c8243e7a6f877cb0017b1a5/users/2fefb68660aa47358a944f9f7840c10a/roles/983be40bdab04c478eaa90c6b6a46b14` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:40:56.265717 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 932/1852] 10.222.0.57 () {58 vars in 1121 bytes} [Fri Jul 1 16:40:56 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 16:40:56.269947 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22142167-b956-418a-ae26-dd77efce0316 admin admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:40:56.282263 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b66185-ba1f-4b77-b91a-acaed9ede2ca admin admin] RBAC: Authorizing `identity:create_grant(project_id=89de13e71c8243e7a6f877cb0017b1a5, user_id=2fefb68660aa47358a944f9f7840c10a, role_id=983be40bdab04c478eaa90c6b6a46b14)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:40:56.282649 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22142167-b956-418a-ae26-dd77efce0316 admin admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:40:56.285690 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b66185-ba1f-4b77-b91a-acaed9ede2ca admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:40:56.288072 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3daa9de9-59ed-4b20-b093-e35e722a7f96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:40:56.288747 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3daa9de9-59ed-4b20-b093-e35e722a7f96 admin admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:40:56.289041 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3daa9de9-59ed-4b20-b093-e35e722a7f96 admin admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:40:56.289323 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3daa9de9-59ed-4b20-b093-e35e722a7f96 admin admin] PATH_INFO: `/v3/roles` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:40:56.290192 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3daa9de9-59ed-4b20-b093-e35e722a7f96 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:40:56.292390 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3daa9de9-59ed-4b20-b093-e35e722a7f96 admin admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:40:56.300842 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 921/1853] 10.222.0.57 () {60 vars in 1482 bytes} [Fri Jul 1 16:40:56 2022] PUT /identity/v3/projects/89de13e71c8243e7a6f877cb0017b1a5/users/2fefb68660aa47358a944f9f7840c10a/roles/983be40bdab04c478eaa90c6b6a46b14 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 01 16:40:56.305499 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 933/1854] 10.222.0.57 () {58 vars in 1121 bytes} [Fri Jul 1 16:40:56 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 01 16:40:56.314252 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435823bf-d834-4e3e-92e9-e8ac64a6924e None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:40:56.314474 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3daa9de9-59ed-4b20-b093-e35e722a7f96 admin admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:40:56.314633 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435823bf-d834-4e3e-92e9-e8ac64a6924e None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:40:56.314897 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435823bf-d834-4e3e-92e9-e8ac64a6924e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:40:56.326828 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3daa9de9-59ed-4b20-b093-e35e722a7f96 admin admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:40:56.348706 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-435823bf-d834-4e3e-92e9-e8ac64a6924e None None] MFA Rules not processed for user `2fefb68660aa47358a944f9f7840c10a`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:40:56.349397 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-435823bf-d834-4e3e-92e9-e8ac64a6924e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:40:56.353808 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:40:56.505175 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b07419-462d-4e6d-9658-5cf601aa506e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:40:56.506115 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b07419-462d-4e6d-9658-5cf601aa506e admin admin] REQUEST_METHOD: `PUT` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:40:56.506504 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b07419-462d-4e6d-9658-5cf601aa506e admin admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:40:56.506743 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b07419-462d-4e6d-9658-5cf601aa506e admin admin] PATH_INFO: `/v3/projects/66b181d4ba3a460d82c817c7cd6ccf20/users/6d56cf849dc84b60981f3cbd81f23ab2/roles/983be40bdab04c478eaa90c6b6a46b14` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:40:56.527970 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b07419-462d-4e6d-9658-5cf601aa506e admin admin] RBAC: Authorizing `identity:create_grant(project_id=66b181d4ba3a460d82c817c7cd6ccf20, user_id=6d56cf849dc84b60981f3cbd81f23ab2, role_id=983be40bdab04c478eaa90c6b6a46b14)` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:40:56.531049 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b07419-462d-4e6d-9658-5cf601aa506e admin admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:40:56.552946 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 934/1855] 10.222.0.57 () {60 vars in 1482 bytes} [Fri Jul 1 16:40:56 2022] PUT /identity/v3/projects/66b181d4ba3a460d82c817c7cd6ccf20/users/6d56cf849dc84b60981f3cbd81f23ab2/roles/983be40bdab04c478eaa90c6b6a46b14 => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 16:40:56.566563 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c10994-6181-4b83-8e52-d48d64e645ed None None] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:40:56.566857 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c10994-6181-4b83-8e52-d48d64e645ed None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:40:56.567170 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c10994-6181-4b83-8e52-d48d64e645ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:40:56.588681 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:40:56.630775 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.auth.core [None req-46c10994-6181-4b83-8e52-d48d64e645ed None None] MFA Rules not processed for user `6d56cf849dc84b60981f3cbd81f23ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=80312) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:40:56.631297 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 922/1856] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 1 16:40:56 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 316 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 16:40:56.632026 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-46c10994-6181-4b83-8e52-d48d64e645ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:40:56.632415 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435823bf-d834-4e3e-92e9-e8ac64a6924e None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:40:56.666553 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435823bf-d834-4e3e-92e9-e8ac64a6924e None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:40:56.779033 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 935/1857] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 1 16:40:56 2022] POST /identity/v3/auth/tokens => generated 3226 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 16:40:56.850596 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c10994-6181-4b83-8e52-d48d64e645ed None None] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:40:56.854567 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-46c10994-6181-4b83-8e52-d48d64e645ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:40:56.857627 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4a21d6-fd14-4d58-a220-1aa5fea154eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:40:56.859025 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4a21d6-fd14-4d58-a220-1aa5fea154eb admin admin] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:40:56.859396 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4a21d6-fd14-4d58-a220-1aa5fea154eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:40:56.859754 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4a21d6-fd14-4d58-a220-1aa5fea154eb admin admin] PATH_INFO: `/v3/projects` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:40:56.860637 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4a21d6-fd14-4d58-a220-1aa5fea154eb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:40:56.864065 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4a21d6-fd14-4d58-a220-1aa5fea154eb admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:40:56.911276 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 923/1858] 10.222.0.57 () {60 vars in 1152 bytes} [Fri Jul 1 16:40:56 2022] POST /identity/v3/projects => generated 367 bytes in 279 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 16:40:56.916049 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4a21d6-fd14-4d58-a220-1aa5fea154eb admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:40:56.983615 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c10994-6181-4b83-8e52-d48d64e645ed None None] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:40:57.057085 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:40:57.076921 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf9bac2-bf9e-4e86-bfb8-ac23edf0f788 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbddf971a2354bbdbb9da1d63f3abf88', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0788e6725a84e57a4174284c1845755', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:40:57.077656 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf9bac2-bf9e-4e86-bfb8-ac23edf0f788 service cinder] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:40:57.077943 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf9bac2-bf9e-4e86-bfb8-ac23edf0f788 service cinder] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:40:57.083301 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf9bac2-bf9e-4e86-bfb8-ac23edf0f788 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:40:57.116849 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4a21d6-fd14-4d58-a220-1aa5fea154eb admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:40:57.133697 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b8623f-890c-447e-852f-298b71ee9ef2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:40:57.135008 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b8623f-890c-447e-852f-298b71ee9ef2 admin admin] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:40:57.137365 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b8623f-890c-447e-852f-298b71ee9ef2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:40:57.137886 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b8623f-890c-447e-852f-298b71ee9ef2 admin admin] PATH_INFO: `/v3/users` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:40:57.139983 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b8623f-890c-447e-852f-298b71ee9ef2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:40:57.142165 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b8623f-890c-447e-852f-298b71ee9ef2 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:40:57.199230 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 924/1859] 10.222.0.57 () {60 vars in 1143 bytes} [Fri Jul 1 16:40:56 2022] POST /identity/v3/users => generated 290 bytes in 284 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 16:40:57.202895 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b8623f-890c-447e-852f-298b71ee9ef2 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:40:57.226701 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b8623f-890c-447e-852f-298b71ee9ef2 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:40:57.241367 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7032ad-a22f-4b30-8bb4-49c3be837970 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:40:57.242290 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7032ad-a22f-4b30-8bb4-49c3be837970 admin admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:40:57.242760 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7032ad-a22f-4b30-8bb4-49c3be837970 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:40:57.243233 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7032ad-a22f-4b30-8bb4-49c3be837970 admin admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:40:57.244928 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc7032ad-a22f-4b30-8bb4-49c3be837970 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:40:57.246981 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc7032ad-a22f-4b30-8bb4-49c3be837970 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:40:57.269964 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 925/1860] 10.222.0.57 () {58 vars in 1121 bytes} [Fri Jul 1 16:40:57 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 01 16:40:57.271384 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf9bac2-bf9e-4e86-bfb8-ac23edf0f788 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:40:57.273753 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7032ad-a22f-4b30-8bb4-49c3be837970 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:40:57.275897 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf9bac2-bf9e-4e86-bfb8-ac23edf0f788 service cinder] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:40:57.304725 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7032ad-a22f-4b30-8bb4-49c3be837970 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:40:57.309424 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 936/1861] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 1 16:40:56 2022] GET /identity/v3/auth/tokens => generated 3226 bytes in 459 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 16:40:57.334053 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf9bac2-bf9e-4e86-bfb8-ac23edf0f788 service cinder] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:40:57.343684 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ec58fd-efe4-4dcd-a316-9008d0d661cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:40:57.347305 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ec58fd-efe4-4dcd-a316-9008d0d661cc admin admin] REQUEST_METHOD: `PUT` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:40:57.347846 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ec58fd-efe4-4dcd-a316-9008d0d661cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:40:57.348841 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ec58fd-efe4-4dcd-a316-9008d0d661cc admin admin] PATH_INFO: `/v3/projects/0ea969a45167484fb43be6f922feec15/users/3a4041b57b9c4a1795bef1c741405414/roles/84ee739146934c1ab32c501ff8093644` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:40:57.366997 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf9bac2-bf9e-4e86-bfb8-ac23edf0f788 service cinder] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:40:57.379569 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa2ac2f-4e35-4ac1-8cd0-9f06dd05148a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:40:57.381693 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa2ac2f-4e35-4ac1-8cd0-9f06dd05148a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:40:57.382252 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa2ac2f-4e35-4ac1-8cd0-9f06dd05148a admin admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:40:57.382677 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa2ac2f-4e35-4ac1-8cd0-9f06dd05148a admin admin] PATH_INFO: `/v3/users/6d56cf849dc84b60981f3cbd81f23ab2` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:40:57.385439 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa2ac2f-4e35-4ac1-8cd0-9f06dd05148a admin admin] RBAC: Authorizing `identity:delete_user(user_id=6d56cf849dc84b60981f3cbd81f23ab2)` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:40:57.387728 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa2ac2f-4e35-4ac1-8cd0-9f06dd05148a admin admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:40:57.393935 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ec58fd-efe4-4dcd-a316-9008d0d661cc admin admin] RBAC: Authorizing `identity:create_grant(project_id=0ea969a45167484fb43be6f922feec15, user_id=3a4041b57b9c4a1795bef1c741405414, role_id=84ee739146934c1ab32c501ff8093644)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:40:57.400231 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ec58fd-efe4-4dcd-a316-9008d0d661cc admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:40:57.441553 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 926/1862] 10.222.0.57 () {60 vars in 1482 bytes} [Fri Jul 1 16:40:57 2022] PUT /identity/v3/projects/0ea969a45167484fb43be6f922feec15/users/3a4041b57b9c4a1795bef1c741405414/roles/84ee739146934c1ab32c501ff8093644 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 16:40:57.444827 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ec58fd-efe4-4dcd-a316-9008d0d661cc admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:40:57.453211 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-d9ec58fd-efe4-4dcd-a316-9008d0d661cc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:40:57.509028 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-dfa2ac2f-4e35-4ac1-8cd0-9f06dd05148a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6d56cf849dc84b60981f3cbd81f23ab2'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:40:57.528152 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:40:57.532577 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-dfa2ac2f-4e35-4ac1-8cd0-9f06dd05148a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6d56cf849dc84b60981f3cbd81f23ab2'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:40:57.546691 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-dfa2ac2f-4e35-4ac1-8cd0-9f06dd05148a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6d56cf849dc84b60981f3cbd81f23ab2'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:40:57.547519 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-dfa2ac2f-4e35-4ac1-8cd0-9f06dd05148a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6d56cf849dc84b60981f3cbd81f23ab2'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:40:57.548972 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-dfa2ac2f-4e35-4ac1-8cd0-9f06dd05148a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6d56cf849dc84b60981f3cbd81f23ab2'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:40:57.575230 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 937/1863] 10.222.0.57 () {58 vars in 1223 bytes} [Fri Jul 1 16:40:57 2022] DELETE /identity/v3/users/6d56cf849dc84b60981f3cbd81f23ab2 => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 16:40:57.576127 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa2ac2f-4e35-4ac1-8cd0-9f06dd05148a admin admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:40:57.579112 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-dfa2ac2f-4e35-4ac1-8cd0-9f06dd05148a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:40:57.615854 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ec58fd-efe4-4dcd-a316-9008d0d661cc admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:40:57.699730 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8ec85d-1943-4d07-bd04-2b3f1f557a39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:40:57.701986 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8ec85d-1943-4d07-bd04-2b3f1f557a39 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:40:57.702734 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8ec85d-1943-4d07-bd04-2b3f1f557a39 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:40:57.703241 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8ec85d-1943-4d07-bd04-2b3f1f557a39 admin admin] PATH_INFO: `/v3/users/be0bed3de52945dab254529167d4c492` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:40:57.705151 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c8ec85d-1943-4d07-bd04-2b3f1f557a39 admin admin] RBAC: Authorizing `identity:delete_user(user_id=be0bed3de52945dab254529167d4c492)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:40:57.716925 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c8ec85d-1943-4d07-bd04-2b3f1f557a39 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:40:57.723829 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa2ac2f-4e35-4ac1-8cd0-9f06dd05148a admin admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:40:57.729843 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e556726-c7d5-4f06-a5d2-1145b3c661d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:40:57.730888 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e556726-c7d5-4f06-a5d2-1145b3c661d8 admin admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:40:57.731425 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e556726-c7d5-4f06-a5d2-1145b3c661d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:40:57.731719 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e556726-c7d5-4f06-a5d2-1145b3c661d8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:40:57.732907 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e556726-c7d5-4f06-a5d2-1145b3c661d8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:40:57.735757 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e556726-c7d5-4f06-a5d2-1145b3c661d8 admin admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:40:57.751613 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 938/1864] 10.222.0.57 () {58 vars in 1121 bytes} [Fri Jul 1 16:40:57 2022] GET /identity/v3/roles => generated 1420 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 16:40:57.754464 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b392ba61-1539-4e09-812c-7e4de03034e5 None None] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:40:57.754769 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b392ba61-1539-4e09-812c-7e4de03034e5 None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:40:57.755108 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b392ba61-1539-4e09-812c-7e4de03034e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:40:57.785721 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:40:57.802859 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.auth.core [None req-b392ba61-1539-4e09-812c-7e4de03034e5 None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80312) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:40:57.804395 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-b392ba61-1539-4e09-812c-7e4de03034e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:40:57.808940 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-6c8ec85d-1943-4d07-bd04-2b3f1f557a39 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'be0bed3de52945dab254529167d4c492'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:40:57.809897 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-6c8ec85d-1943-4d07-bd04-2b3f1f557a39 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'be0bed3de52945dab254529167d4c492'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:40:57.810782 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-6c8ec85d-1943-4d07-bd04-2b3f1f557a39 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'be0bed3de52945dab254529167d4c492'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:40:57.817724 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 939/1865] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 1 16:40:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 65 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 16:40:57.818685 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b392ba61-1539-4e09-812c-7e4de03034e5 None None] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:40:57.835664 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-6c8ec85d-1943-4d07-bd04-2b3f1f557a39 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'be0bed3de52945dab254529167d4c492'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:40:57.849643 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b392ba61-1539-4e09-812c-7e4de03034e5 None None] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:40:57.862653 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-6c8ec85d-1943-4d07-bd04-2b3f1f557a39 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'be0bed3de52945dab254529167d4c492'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:40:57.866600 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692e21d7-d5b4-4665-b86d-1633c07b1f4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:40:57.867642 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692e21d7-d5b4-4665-b86d-1633c07b1f4e admin admin] REQUEST_METHOD: `PUT` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:40:57.868004 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692e21d7-d5b4-4665-b86d-1633c07b1f4e admin admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:40:57.868344 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692e21d7-d5b4-4665-b86d-1633c07b1f4e admin admin] PATH_INFO: `/v3/projects/0ea969a45167484fb43be6f922feec15/users/3a4041b57b9c4a1795bef1c741405414/roles/84ee739146934c1ab32c501ff8093644` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:40:57.871084 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-692e21d7-d5b4-4665-b86d-1633c07b1f4e admin admin] RBAC: Authorizing `identity:create_grant(project_id=0ea969a45167484fb43be6f922feec15, user_id=3a4041b57b9c4a1795bef1c741405414, role_id=84ee739146934c1ab32c501ff8093644)` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:40:57.873799 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-692e21d7-d5b4-4665-b86d-1633c07b1f4e admin admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:40:57.878483 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 927/1866] 10.222.0.57 () {58 vars in 1223 bytes} [Fri Jul 1 16:40:57 2022] DELETE /identity/v3/users/be0bed3de52945dab254529167d4c492 => generated 0 bytes in 436 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 16:40:57.879492 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8ec85d-1943-4d07-bd04-2b3f1f557a39 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:40:57.880914 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-6c8ec85d-1943-4d07-bd04-2b3f1f557a39 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:40:57.892658 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 940/1867] 10.222.0.57 () {60 vars in 1482 bytes} [Fri Jul 1 16:40:57 2022] PUT /identity/v3/projects/0ea969a45167484fb43be6f922feec15/users/3a4041b57b9c4a1795bef1c741405414/roles/84ee739146934c1ab32c501ff8093644 => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 16:40:57.895864 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff712bd-c6ab-4187-8181-6127d366f2e4 None None] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:40:57.896376 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff712bd-c6ab-4187-8181-6127d366f2e4 None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:40:57.896804 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff712bd-c6ab-4187-8181-6127d366f2e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:40:57.919405 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.auth.core [None req-aff712bd-c6ab-4187-8181-6127d366f2e4 None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80312) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:40:57.920631 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-aff712bd-c6ab-4187-8181-6127d366f2e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:40:57.946558 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8ec85d-1943-4d07-bd04-2b3f1f557a39 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:40:57.973281 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:40:57.993472 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 941/1868] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 1 16:40:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 16:40:57.995270 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371d2d36-593c-4f8b-8b28-aa6ffa5c52c7 None None] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:40:57.995567 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371d2d36-593c-4f8b-8b28-aa6ffa5c52c7 None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:40:57.995846 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371d2d36-593c-4f8b-8b28-aa6ffa5c52c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:40:58.014106 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f2e3ac-a34a-4cbc-ae9a-cff020a90415 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ed200ce29684c5f80354fa9a8638fe2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0788e6725a84e57a4174284c1845755', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:40:58.014974 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f2e3ac-a34a-4cbc-ae9a-cff020a90415 service neutron] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:40:58.015422 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f2e3ac-a34a-4cbc-ae9a-cff020a90415 service neutron] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:40:58.015846 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f2e3ac-a34a-4cbc-ae9a-cff020a90415 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:40:58.018572 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-50f2e3ac-a34a-4cbc-ae9a-cff020a90415 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:40:58.026925 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.auth.core [None req-371d2d36-593c-4f8b-8b28-aa6ffa5c52c7 None None] MFA Rules not processed for user `3a4041b57b9c4a1795bef1c741405414`. Rule list: `[]` (Enabled: `True`). {{(pid=80312) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:40:58.028262 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-371d2d36-593c-4f8b-8b28-aa6ffa5c52c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:40:58.035390 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f2e3ac-a34a-4cbc-ae9a-cff020a90415 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:40:58.038108 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f2e3ac-a34a-4cbc-ae9a-cff020a90415 service neutron] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:40:58.054518 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 928/1869] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 1 16:40:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 175 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 16:40:58.055609 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f2e3ac-a34a-4cbc-ae9a-cff020a90415 service neutron] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:40:58.069866 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f2e3ac-a34a-4cbc-ae9a-cff020a90415 service neutron] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:40:58.075724 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b0b319-1c58-47a8-8813-1b4a613451f5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ed200ce29684c5f80354fa9a8638fe2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0788e6725a84e57a4174284c1845755', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:40:58.076582 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b0b319-1c58-47a8-8813-1b4a613451f5 service neutron] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:40:58.077008 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b0b319-1c58-47a8-8813-1b4a613451f5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:40:58.077423 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b0b319-1c58-47a8-8813-1b4a613451f5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:40:58.093325 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b0b319-1c58-47a8-8813-1b4a613451f5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:40:58.094980 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b0b319-1c58-47a8-8813-1b4a613451f5 service neutron] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:40:58.114687 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 929/1870] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 1 16:40:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 16:40:58.145857 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 942/1871] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 1 16:40:57 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 16:40:58.170239 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b0b319-1c58-47a8-8813-1b4a613451f5 service neutron] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:40:58.171920 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-53b0b319-1c58-47a8-8813-1b4a613451f5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:40:58.247035 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b0b319-1c58-47a8-8813-1b4a613451f5 service neutron] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:40:58.256833 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7d3ce8-3ea9-4e7a-838e-c4402e33d704 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbddf971a2354bbdbb9da1d63f3abf88', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0788e6725a84e57a4174284c1845755', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:40:58.257441 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7d3ce8-3ea9-4e7a-838e-c4402e33d704 service cinder] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:40:58.258514 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7d3ce8-3ea9-4e7a-838e-c4402e33d704 service cinder] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:40:58.258804 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7d3ce8-3ea9-4e7a-838e-c4402e33d704 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:40:58.271005 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a7d3ce8-3ea9-4e7a-838e-c4402e33d704 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:40:58.272464 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a7d3ce8-3ea9-4e7a-838e-c4402e33d704 service cinder] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:40:58.288162 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 930/1872] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 1 16:40:58 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 16:40:58.355834 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371d2d36-593c-4f8b-8b28-aa6ffa5c52c7 None None] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:40:58.372887 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371d2d36-593c-4f8b-8b28-aa6ffa5c52c7 None None] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:40:58.381644 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422d7daa-4373-4410-a839-332828addb4e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbddf971a2354bbdbb9da1d63f3abf88', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0788e6725a84e57a4174284c1845755', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:40:58.382460 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422d7daa-4373-4410-a839-332828addb4e service cinder] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:40:58.382746 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422d7daa-4373-4410-a839-332828addb4e service cinder] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:40:58.383106 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422d7daa-4373-4410-a839-332828addb4e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:40:58.386897 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-422d7daa-4373-4410-a839-332828addb4e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:40:58.513710 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-422d7daa-4373-4410-a839-332828addb4e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:40:58.515374 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-422d7daa-4373-4410-a839-332828addb4e service cinder] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:40:58.568267 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 943/1873] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 1 16:40:58 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 213 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 16:40:59.767498 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7d3ce8-3ea9-4e7a-838e-c4402e33d704 service cinder] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:40:59.771136 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-1a7d3ce8-3ea9-4e7a-838e-c4402e33d704 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:40:59.778014 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422d7daa-4373-4410-a839-332828addb4e service cinder] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:40:59.779460 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-422d7daa-4373-4410-a839-332828addb4e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:40:59.795023 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422d7daa-4373-4410-a839-332828addb4e service cinder] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:40:59.795539 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7d3ce8-3ea9-4e7a-838e-c4402e33d704 service cinder] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:40:59.800985 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2652e42-57a7-429c-a62c-6f291ea3011d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:40:59.802193 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2652e42-57a7-429c-a62c-6f291ea3011d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:40:59.802593 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2652e42-57a7-429c-a62c-6f291ea3011d admin admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:40:59.802949 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2652e42-57a7-429c-a62c-6f291ea3011d admin admin] PATH_INFO: `/v3/projects/66b181d4ba3a460d82c817c7cd6ccf20` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:40:59.804319 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2652e42-57a7-429c-a62c-6f291ea3011d admin admin] RBAC: Authorizing `identity:delete_project(project_id=66b181d4ba3a460d82c817c7cd6ccf20)` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:40:59.806780 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2652e42-57a7-429c-a62c-6f291ea3011d admin admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:40:59.810434 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:40:59.816664 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0605866-5613-45cc-be74-42fea54a1a39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:40:59.819187 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0605866-5613-45cc-be74-42fea54a1a39 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:40:59.819937 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0605866-5613-45cc-be74-42fea54a1a39 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:40:59.820389 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0605866-5613-45cc-be74-42fea54a1a39 admin admin] PATH_INFO: `/v3/projects/943fef40af6f42a3bdde3775c2ca0297` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:40:59.828732 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0605866-5613-45cc-be74-42fea54a1a39 admin admin] RBAC: Authorizing `identity:delete_project(project_id=943fef40af6f42a3bdde3775c2ca0297)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:40:59.836292 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0605866-5613-45cc-be74-42fea54a1a39 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:40:59.902385 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-e2652e42-57a7-429c-a62c-6f291ea3011d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '66b181d4ba3a460d82c817c7cd6ccf20', 'request_id': 'req-e2652e42-57a7-429c-a62c-6f291ea3011d'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:40:59.916380 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-e2652e42-57a7-429c-a62c-6f291ea3011d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '66b181d4ba3a460d82c817c7cd6ccf20', 'request_id': 'req-e2652e42-57a7-429c-a62c-6f291ea3011d'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:40:59.926607 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-e2652e42-57a7-429c-a62c-6f291ea3011d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '66b181d4ba3a460d82c817c7cd6ccf20', 'request_id': 'req-e2652e42-57a7-429c-a62c-6f291ea3011d'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:40:59.942069 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-e2652e42-57a7-429c-a62c-6f291ea3011d admin admin] The token cache is being invalidate because project 66b181d4ba3a460d82c817c7cd6ccf20 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80312) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 01 16:40:59.942496 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-e2652e42-57a7-429c-a62c-6f291ea3011d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:40:59.943546 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-e2652e42-57a7-429c-a62c-6f291ea3011d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:40:59.945438 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 944/1874] 10.222.0.57 () {58 vars in 1232 bytes} [Fri Jul 1 16:40:59 2022] DELETE /identity/v3/projects/66b181d4ba3a460d82c817c7cd6ccf20 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 16:40:59.949786 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd28e018-033d-4e81-a979-d6e20d70cf56 None None] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:40:59.950154 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd28e018-033d-4e81-a979-d6e20d70cf56 None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:40:59.950534 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd28e018-033d-4e81-a979-d6e20d70cf56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:40:59.979727 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-e0605866-5613-45cc-be74-42fea54a1a39 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '943fef40af6f42a3bdde3775c2ca0297', 'request_id': 'req-e0605866-5613-45cc-be74-42fea54a1a39'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:40:59.991942 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-e0605866-5613-45cc-be74-42fea54a1a39 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '943fef40af6f42a3bdde3775c2ca0297', 'request_id': 'req-e0605866-5613-45cc-be74-42fea54a1a39'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:41:00.000436 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-e0605866-5613-45cc-be74-42fea54a1a39 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '943fef40af6f42a3bdde3775c2ca0297', 'request_id': 'req-e0605866-5613-45cc-be74-42fea54a1a39'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:41:00.006080 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.auth.core [None req-bd28e018-033d-4e81-a979-d6e20d70cf56 None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80312) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:41:00.008576 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-bd28e018-033d-4e81-a979-d6e20d70cf56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:41:00.031358 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-e0605866-5613-45cc-be74-42fea54a1a39 admin admin] The token cache is being invalidate because project 943fef40af6f42a3bdde3775c2ca0297 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80311) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 01 16:41:00.033295 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-e0605866-5613-45cc-be74-42fea54a1a39 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:41:00.035366 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-e0605866-5613-45cc-be74-42fea54a1a39 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:41:00.039511 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 931/1875] 10.222.0.57 () {58 vars in 1232 bytes} [Fri Jul 1 16:40:59 2022] DELETE /identity/v3/projects/943fef40af6f42a3bdde3775c2ca0297 => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 01 16:41:00.059281 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0605866-5613-45cc-be74-42fea54a1a39 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:41:00.064110 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-e0605866-5613-45cc-be74-42fea54a1a39 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:41:00.117712 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 945/1876] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 1 16:40:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 16:41:00.119945 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9b88ed-637a-4639-9724-6b289d15ca27 None None] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:41:00.120249 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9b88ed-637a-4639-9724-6b289d15ca27 None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:41:00.125284 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9b88ed-637a-4639-9724-6b289d15ca27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:41:00.170397 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.auth.core [None req-1f9b88ed-637a-4639-9724-6b289d15ca27 None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80312) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:41:00.176964 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-1f9b88ed-637a-4639-9724-6b289d15ca27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:41:00.191685 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:41:00.216963 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 946/1877] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 1 16:41:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 16:41:00.222657 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9b88ed-637a-4639-9724-6b289d15ca27 None None] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:41:00.243252 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0605866-5613-45cc-be74-42fea54a1a39 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:41:00.245450 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9b88ed-637a-4639-9724-6b289d15ca27 None None] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:41:00.248635 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0aa74d-b02e-4657-9a79-ed39c9ba76bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:41:00.250425 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0aa74d-b02e-4657-9a79-ed39c9ba76bb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:41:00.250719 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0aa74d-b02e-4657-9a79-ed39c9ba76bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:41:00.250999 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0aa74d-b02e-4657-9a79-ed39c9ba76bb admin admin] PATH_INFO: `/v3/users/6a5d2d3d9b004b4f91d51609d04b40d7` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:41:00.253615 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa0aa74d-b02e-4657-9a79-ed39c9ba76bb admin admin] RBAC: Authorizing `identity:delete_user(user_id=6a5d2d3d9b004b4f91d51609d04b40d7)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:41:00.257042 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa0aa74d-b02e-4657-9a79-ed39c9ba76bb admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:41:00.274679 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c20a9e-8150-408f-8e5e-eb5a208684ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:41:00.275519 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c20a9e-8150-408f-8e5e-eb5a208684ce admin admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:41:00.276055 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c20a9e-8150-408f-8e5e-eb5a208684ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:41:00.276434 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c20a9e-8150-408f-8e5e-eb5a208684ce admin admin] PATH_INFO: `/v3/domains` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:41:00.278299 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c20a9e-8150-408f-8e5e-eb5a208684ce admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80312) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:41:00.278862 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c20a9e-8150-408f-8e5e-eb5a208684ce admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:41:00.283833 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c20a9e-8150-408f-8e5e-eb5a208684ce admin admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:41:00.337616 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 947/1878] 10.222.0.57 () {58 vars in 1152 bytes} [Fri Jul 1 16:41:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:41:00.339401 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c20a9e-8150-408f-8e5e-eb5a208684ce admin admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:41:00.372082 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-fa0aa74d-b02e-4657-9a79-ed39c9ba76bb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6a5d2d3d9b004b4f91d51609d04b40d7'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:41:00.373020 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-fa0aa74d-b02e-4657-9a79-ed39c9ba76bb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6a5d2d3d9b004b4f91d51609d04b40d7'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:41:00.376809 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-fa0aa74d-b02e-4657-9a79-ed39c9ba76bb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6a5d2d3d9b004b4f91d51609d04b40d7'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:41:00.377895 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c20a9e-8150-408f-8e5e-eb5a208684ce admin admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:41:00.392360 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8c653b-24ac-4707-a515-01aa9e3a20b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:41:00.393956 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8c653b-24ac-4707-a515-01aa9e3a20b8 admin admin] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:41:00.394375 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8c653b-24ac-4707-a515-01aa9e3a20b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:41:00.394934 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8c653b-24ac-4707-a515-01aa9e3a20b8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:41:00.395942 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce8c653b-24ac-4707-a515-01aa9e3a20b8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:41:00.398472 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce8c653b-24ac-4707-a515-01aa9e3a20b8 admin admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:41:00.399032 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-fa0aa74d-b02e-4657-9a79-ed39c9ba76bb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6a5d2d3d9b004b4f91d51609d04b40d7'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:41:00.442944 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-fa0aa74d-b02e-4657-9a79-ed39c9ba76bb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6a5d2d3d9b004b4f91d51609d04b40d7'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:41:00.465009 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 932/1879] 10.222.0.57 () {58 vars in 1223 bytes} [Fri Jul 1 16:41:00 2022] DELETE /identity/v3/users/6a5d2d3d9b004b4f91d51609d04b40d7 => generated 0 bytes in 421 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 16:41:00.499855 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0aa74d-b02e-4657-9a79-ed39c9ba76bb admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:41:00.502871 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-fa0aa74d-b02e-4657-9a79-ed39c9ba76bb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:41:00.515194 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 948/1880] 10.222.0.57 () {60 vars in 1152 bytes} [Fri Jul 1 16:41:00 2022] POST /identity/v3/projects => generated 377 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 16:41:00.651725 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0aa74d-b02e-4657-9a79-ed39c9ba76bb admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:41:00.922295 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710e161a-ccd5-4642-862a-44a2044fc651 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:41:00.923927 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710e161a-ccd5-4642-862a-44a2044fc651 admin admin] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:41:00.930061 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710e161a-ccd5-4642-862a-44a2044fc651 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:41:00.930677 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710e161a-ccd5-4642-862a-44a2044fc651 admin admin] PATH_INFO: `/v3/users` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:41:00.932107 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-710e161a-ccd5-4642-862a-44a2044fc651 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:41:00.935508 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-710e161a-ccd5-4642-862a-44a2044fc651 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:41:00.950860 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:41:00.982981 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 933/1881] 10.222.0.57 () {60 vars in 1143 bytes} [Fri Jul 1 16:41:00 2022] POST /identity/v3/users => generated 289 bytes in 483 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 16:41:00.984998 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8c653b-24ac-4707-a515-01aa9e3a20b8 admin admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:41:01.003281 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8c653b-24ac-4707-a515-01aa9e3a20b8 admin admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:41:01.011486 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a56909-1f77-4a35-a67a-e4c8f3a84760 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:41:01.015487 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a56909-1f77-4a35-a67a-e4c8f3a84760 admin admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:41:01.015969 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a56909-1f77-4a35-a67a-e4c8f3a84760 admin admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:41:01.016559 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a56909-1f77-4a35-a67a-e4c8f3a84760 admin admin] PATH_INFO: `/v3/roles` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:41:01.018435 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a56909-1f77-4a35-a67a-e4c8f3a84760 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:41:01.022369 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a56909-1f77-4a35-a67a-e4c8f3a84760 admin admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:41:01.041983 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 949/1882] 10.222.0.57 () {58 vars in 1121 bytes} [Fri Jul 1 16:41:00 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 16:41:01.042835 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710e161a-ccd5-4642-862a-44a2044fc651 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:41:01.060942 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710e161a-ccd5-4642-862a-44a2044fc651 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:41:01.068646 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22381a48-4bab-45c8-acb0-78b6186fe86c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:41:01.069537 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22381a48-4bab-45c8-acb0-78b6186fe86c admin admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:41:01.070018 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22381a48-4bab-45c8-acb0-78b6186fe86c admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:41:01.070392 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22381a48-4bab-45c8-acb0-78b6186fe86c admin admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:41:01.085244 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22381a48-4bab-45c8-acb0-78b6186fe86c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:41:01.100090 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22381a48-4bab-45c8-acb0-78b6186fe86c admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:41:01.141660 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 934/1883] 10.222.0.57 () {58 vars in 1121 bytes} [Fri Jul 1 16:41:01 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 01 16:41:01.147758 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a56909-1f77-4a35-a67a-e4c8f3a84760 admin admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:41:01.169011 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a56909-1f77-4a35-a67a-e4c8f3a84760 admin admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:41:01.181902 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0732ce47-7ff0-43a3-85aa-7c307d0beda0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:41:01.182921 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0732ce47-7ff0-43a3-85aa-7c307d0beda0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:41:01.186741 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0732ce47-7ff0-43a3-85aa-7c307d0beda0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:41:01.187122 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0732ce47-7ff0-43a3-85aa-7c307d0beda0 admin admin] PATH_INFO: `/v3/projects/c30ee4688f3c4f40b62c810dab8c7d82/users/3b430c67c85e437e810d2d967a3af5c0/roles/983be40bdab04c478eaa90c6b6a46b14` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:41:01.218629 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0732ce47-7ff0-43a3-85aa-7c307d0beda0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c30ee4688f3c4f40b62c810dab8c7d82, user_id=3b430c67c85e437e810d2d967a3af5c0, role_id=983be40bdab04c478eaa90c6b6a46b14)` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:41:01.222870 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0732ce47-7ff0-43a3-85aa-7c307d0beda0 admin admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:41:01.248165 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 950/1884] 10.222.0.57 () {60 vars in 1482 bytes} [Fri Jul 1 16:41:01 2022] PUT /identity/v3/projects/c30ee4688f3c4f40b62c810dab8c7d82/users/3b430c67c85e437e810d2d967a3af5c0/roles/983be40bdab04c478eaa90c6b6a46b14 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 16:41:01.260359 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164c3fea-0dd6-46a6-ae0f-a4186697c59c None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:41:01.260669 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164c3fea-0dd6-46a6-ae0f-a4186697c59c None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:41:01.260953 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164c3fea-0dd6-46a6-ae0f-a4186697c59c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:41:01.308033 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-164c3fea-0dd6-46a6-ae0f-a4186697c59c None None] MFA Rules not processed for user `3b430c67c85e437e810d2d967a3af5c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:41:01.309806 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-164c3fea-0dd6-46a6-ae0f-a4186697c59c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:41:01.468521 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 935/1885] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 1 16:41:01 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 16:41:01.473603 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0732ce47-7ff0-43a3-85aa-7c307d0beda0 admin admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:41:01.500045 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:41:01.569540 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0732ce47-7ff0-43a3-85aa-7c307d0beda0 admin admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:41:01.576253 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fda171d-f004-4a06-85c7-fc5b68283608 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:41:01.577105 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fda171d-f004-4a06-85c7-fc5b68283608 admin admin] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:41:01.577488 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fda171d-f004-4a06-85c7-fc5b68283608 admin admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:41:01.577872 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fda171d-f004-4a06-85c7-fc5b68283608 admin admin] PATH_INFO: `/v3/projects` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:41:01.579059 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fda171d-f004-4a06-85c7-fc5b68283608 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:41:01.582391 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fda171d-f004-4a06-85c7-fc5b68283608 admin admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:41:01.625876 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 951/1886] 10.222.0.57 () {60 vars in 1152 bytes} [Fri Jul 1 16:41:01 2022] POST /identity/v3/projects => generated 377 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 16:41:01.630584 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164c3fea-0dd6-46a6-ae0f-a4186697c59c None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:41:01.644557 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164c3fea-0dd6-46a6-ae0f-a4186697c59c None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:41:01.744362 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845edb22-582f-4ac2-9d9a-123ab7dab3e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:41:01.745065 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845edb22-582f-4ac2-9d9a-123ab7dab3e7 admin admin] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:41:01.745356 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845edb22-582f-4ac2-9d9a-123ab7dab3e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:41:01.745642 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845edb22-582f-4ac2-9d9a-123ab7dab3e7 admin admin] PATH_INFO: `/v3/users` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:41:01.746465 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-845edb22-582f-4ac2-9d9a-123ab7dab3e7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:41:01.748446 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-845edb22-582f-4ac2-9d9a-123ab7dab3e7 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:41:01.781455 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 936/1887] 10.222.0.57 () {60 vars in 1143 bytes} [Fri Jul 1 16:41:01 2022] POST /identity/v3/users => generated 289 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 16:41:01.781597 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fda171d-f004-4a06-85c7-fc5b68283608 admin admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:41:01.793265 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fda171d-f004-4a06-85c7-fc5b68283608 admin admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:41:01.800682 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed13e03b-4256-4808-8b50-b58975393109 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:41:01.803359 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed13e03b-4256-4808-8b50-b58975393109 admin admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:41:01.803359 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed13e03b-4256-4808-8b50-b58975393109 admin admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:41:01.803359 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed13e03b-4256-4808-8b50-b58975393109 admin admin] PATH_INFO: `/v3/roles` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:41:01.803359 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed13e03b-4256-4808-8b50-b58975393109 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:41:01.806784 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed13e03b-4256-4808-8b50-b58975393109 admin admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:41:01.832120 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 952/1888] 10.222.0.57 () {58 vars in 1121 bytes} [Fri Jul 1 16:41:01 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 16:41:01.835147 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845edb22-582f-4ac2-9d9a-123ab7dab3e7 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:41:01.861704 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845edb22-582f-4ac2-9d9a-123ab7dab3e7 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:41:01.875488 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2fed29f-7dac-4cdc-b41b-a73414be1935 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:41:01.876453 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fed29f-7dac-4cdc-b41b-a73414be1935 admin admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:41:01.876781 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fed29f-7dac-4cdc-b41b-a73414be1935 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:41:01.877744 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fed29f-7dac-4cdc-b41b-a73414be1935 admin admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:41:01.880153 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2fed29f-7dac-4cdc-b41b-a73414be1935 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:41:01.902955 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2fed29f-7dac-4cdc-b41b-a73414be1935 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:41:01.937889 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 937/1889] 10.222.0.57 () {58 vars in 1121 bytes} [Fri Jul 1 16:41:01 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 16:41:01.944143 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed13e03b-4256-4808-8b50-b58975393109 admin admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:41:01.962843 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed13e03b-4256-4808-8b50-b58975393109 admin admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:41:01.982181 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e52501-9b85-4b8c-9538-e8d14c52aff0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:41:01.983322 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e52501-9b85-4b8c-9538-e8d14c52aff0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:41:01.983770 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e52501-9b85-4b8c-9538-e8d14c52aff0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:41:01.984208 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e52501-9b85-4b8c-9538-e8d14c52aff0 admin admin] PATH_INFO: `/v3/projects/1395a12992e9423286ddc2ad5265c260/users/e83a9e97966d493ba56184049aa097d5/roles/983be40bdab04c478eaa90c6b6a46b14` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:41:02.016349 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e52501-9b85-4b8c-9538-e8d14c52aff0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1395a12992e9423286ddc2ad5265c260, user_id=e83a9e97966d493ba56184049aa097d5, role_id=983be40bdab04c478eaa90c6b6a46b14)` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:41:02.019326 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e52501-9b85-4b8c-9538-e8d14c52aff0 admin admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:41:02.076285 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 953/1890] 10.222.0.57 () {60 vars in 1482 bytes} [Fri Jul 1 16:41:01 2022] PUT /identity/v3/projects/1395a12992e9423286ddc2ad5265c260/users/e83a9e97966d493ba56184049aa097d5/roles/983be40bdab04c478eaa90c6b6a46b14 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 16:41:02.081301 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc3ef04-6b68-491f-9171-bc246f24743f None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:41:02.082242 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc3ef04-6b68-491f-9171-bc246f24743f None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:41:02.082716 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc3ef04-6b68-491f-9171-bc246f24743f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:41:02.152763 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-1fc3ef04-6b68-491f-9171-bc246f24743f None None] MFA Rules not processed for user `e83a9e97966d493ba56184049aa097d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:41:02.154382 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-1fc3ef04-6b68-491f-9171-bc246f24743f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:41:02.172307 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e52501-9b85-4b8c-9538-e8d14c52aff0 admin admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:41:02.176240 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-38e52501-9b85-4b8c-9538-e8d14c52aff0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:41:02.242604 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:INSERT=5,keystone:SELECT=95 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:41:02.321790 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 938/1891] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 1 16:41:02 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 243 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 16:41:02.326282 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc3ef04-6b68-491f-9171-bc246f24743f None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:41:02.458866 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e52501-9b85-4b8c-9538-e8d14c52aff0 admin admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:41:02.473853 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18779c49-c2bf-4223-8a2a-854440741637 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:41:02.475070 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18779c49-c2bf-4223-8a2a-854440741637 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:41:02.475700 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18779c49-c2bf-4223-8a2a-854440741637 admin admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:41:02.476275 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18779c49-c2bf-4223-8a2a-854440741637 admin admin] PATH_INFO: `/v3/projects/811c93ee3a4849c5974599f1a03f9188` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:41:02.478148 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc3ef04-6b68-491f-9171-bc246f24743f None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:41:02.479348 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18779c49-c2bf-4223-8a2a-854440741637 admin admin] RBAC: Authorizing `identity:delete_project(project_id=811c93ee3a4849c5974599f1a03f9188)` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:41:02.489112 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852cb49c-e27c-4881-83d1-a2291d2ad435 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:41:02.490227 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852cb49c-e27c-4881-83d1-a2291d2ad435 admin admin] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:41:02.490837 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852cb49c-e27c-4881-83d1-a2291d2ad435 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:41:02.491584 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852cb49c-e27c-4881-83d1-a2291d2ad435 admin admin] PATH_INFO: `/v3/projects` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:41:02.492639 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-852cb49c-e27c-4881-83d1-a2291d2ad435 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:41:02.494212 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18779c49-c2bf-4223-8a2a-854440741637 admin admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:41:02.495251 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-852cb49c-e27c-4881-83d1-a2291d2ad435 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:41:02.536184 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 939/1892] 10.222.0.57 () {60 vars in 1152 bytes} [Fri Jul 1 16:41:02 2022] POST /identity/v3/projects => generated 377 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 16:41:02.544532 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852cb49c-e27c-4881-83d1-a2291d2ad435 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:41:02.643928 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-18779c49-c2bf-4223-8a2a-854440741637 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '811c93ee3a4849c5974599f1a03f9188', 'request_id': 'req-18779c49-c2bf-4223-8a2a-854440741637'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:41:02.684286 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-18779c49-c2bf-4223-8a2a-854440741637 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '811c93ee3a4849c5974599f1a03f9188', 'request_id': 'req-18779c49-c2bf-4223-8a2a-854440741637'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:41:02.697315 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-18779c49-c2bf-4223-8a2a-854440741637 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '811c93ee3a4849c5974599f1a03f9188', 'request_id': 'req-18779c49-c2bf-4223-8a2a-854440741637'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:41:02.703151 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:41:02.703338 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852cb49c-e27c-4881-83d1-a2291d2ad435 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:41:02.743072 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-18779c49-c2bf-4223-8a2a-854440741637 admin admin] The token cache is being invalidate because project 811c93ee3a4849c5974599f1a03f9188 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80312) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 01 16:41:02.743377 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-18779c49-c2bf-4223-8a2a-854440741637 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:41:02.744075 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-18779c49-c2bf-4223-8a2a-854440741637 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:41:02.749980 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 954/1893] 10.222.0.57 () {58 vars in 1232 bytes} [Fri Jul 1 16:41:02 2022] DELETE /identity/v3/projects/811c93ee3a4849c5974599f1a03f9188 => generated 0 bytes in 575 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 16:41:02.751539 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8cec9a-76f5-4ce1-8888-659f3ab68566 None None] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:41:02.751840 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8cec9a-76f5-4ce1-8888-659f3ab68566 None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:41:02.752320 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8cec9a-76f5-4ce1-8888-659f3ab68566 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:41:02.777306 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.auth.core [None req-ba8cec9a-76f5-4ce1-8888-659f3ab68566 None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80312) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:41:02.778732 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-ba8cec9a-76f5-4ce1-8888-659f3ab68566 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:41:02.878841 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c70658-d23e-4b83-b39a-d19fe1d5a32e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:41:02.882563 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c70658-d23e-4b83-b39a-d19fe1d5a32e admin admin] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:41:02.884071 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c70658-d23e-4b83-b39a-d19fe1d5a32e admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:41:02.885065 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c70658-d23e-4b83-b39a-d19fe1d5a32e admin admin] PATH_INFO: `/v3/users` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:41:02.886293 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c70658-d23e-4b83-b39a-d19fe1d5a32e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:41:02.893903 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c70658-d23e-4b83-b39a-d19fe1d5a32e admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:41:02.942647 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:41:02.947130 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 955/1894] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 1 16:41:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 16:41:02.953699 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 940/1895] 10.222.0.57 () {60 vars in 1143 bytes} [Fri Jul 1 16:41:02 2022] POST /identity/v3/users => generated 295 bytes in 416 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 16:41:02.954308 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8cec9a-76f5-4ce1-8888-659f3ab68566 None None] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:41:02.956819 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-ba8cec9a-76f5-4ce1-8888-659f3ab68566 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:41:02.957877 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bf78f5-138b-4689-aae9-c37841da8b72 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:41:02.959631 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bf78f5-138b-4689-aae9-c37841da8b72 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:41:02.960050 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bf78f5-138b-4689-aae9-c37841da8b72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:41:02.987266 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8cec9a-76f5-4ce1-8888-659f3ab68566 None None] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:41:03.003393 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31f3462-e7cc-4fcb-ba64-87e21254da9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:41:03.004173 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31f3462-e7cc-4fcb-ba64-87e21254da9d admin admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:41:03.004467 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31f3462-e7cc-4fcb-ba64-87e21254da9d admin admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:41:03.004749 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31f3462-e7cc-4fcb-ba64-87e21254da9d admin admin] PATH_INFO: `/v3/roles` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:41:03.005496 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c31f3462-e7cc-4fcb-ba64-87e21254da9d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:41:03.007894 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c31f3462-e7cc-4fcb-ba64-87e21254da9d admin admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:41:03.010914 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-d4bf78f5-138b-4689-aae9-c37841da8b72 None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:41:03.015512 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-d4bf78f5-138b-4689-aae9-c37841da8b72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:41:03.038190 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 941/1896] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 1 16:41:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 16:41:03.043031 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 956/1897] 10.222.0.57 () {58 vars in 1121 bytes} [Fri Jul 1 16:41:02 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 01 16:41:03.046481 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4bf78f5-138b-4689-aae9-c37841da8b72 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:41:03.047148 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31f3462-e7cc-4fcb-ba64-87e21254da9d admin admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:41:03.072622 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31f3462-e7cc-4fcb-ba64-87e21254da9d admin admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:41:03.090092 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4bf78f5-138b-4689-aae9-c37841da8b72 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:41:03.094528 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5868b13f-66b3-49b6-b045-f7ad3b16430c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:41:03.095770 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5868b13f-66b3-49b6-b045-f7ad3b16430c admin admin] REQUEST_METHOD: `PUT` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:41:03.096707 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5868b13f-66b3-49b6-b045-f7ad3b16430c admin admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:41:03.097142 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5868b13f-66b3-49b6-b045-f7ad3b16430c admin admin] PATH_INFO: `/v3/projects/10bf42ede27e4c9c8a2faa4dd348fbff/users/2f78e619ff21465ea0d4d6a19e640b6a/roles/84ee739146934c1ab32c501ff8093644` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:41:03.122859 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bcc241b-d9c6-407b-b185-139d66a36be2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:41:03.125103 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcc241b-d9c6-407b-b185-139d66a36be2 admin admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:41:03.127370 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcc241b-d9c6-407b-b185-139d66a36be2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:41:03.130448 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcc241b-d9c6-407b-b185-139d66a36be2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:41:03.134596 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bcc241b-d9c6-407b-b185-139d66a36be2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:41:03.135253 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5868b13f-66b3-49b6-b045-f7ad3b16430c admin admin] RBAC: Authorizing `identity:create_grant(project_id=10bf42ede27e4c9c8a2faa4dd348fbff, user_id=2f78e619ff21465ea0d4d6a19e640b6a, role_id=84ee739146934c1ab32c501ff8093644)` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:41:03.137436 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bcc241b-d9c6-407b-b185-139d66a36be2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:41:03.137984 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5868b13f-66b3-49b6-b045-f7ad3b16430c admin admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:41:03.139961 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bcc241b-d9c6-407b-b185-139d66a36be2 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:41:03.170475 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 957/1898] 10.222.0.57 () {60 vars in 1482 bytes} [Fri Jul 1 16:41:03 2022] PUT /identity/v3/projects/10bf42ede27e4c9c8a2faa4dd348fbff/users/2f78e619ff21465ea0d4d6a19e640b6a/roles/84ee739146934c1ab32c501ff8093644 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 01 16:41:03.174892 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5868b13f-66b3-49b6-b045-f7ad3b16430c admin admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:41:03.211610 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 942/1899] 10.222.0.57 () {58 vars in 1152 bytes} [Fri Jul 1 16:41:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:41:03.212548 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bcc241b-d9c6-407b-b185-139d66a36be2 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:41:03.251209 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bcc241b-d9c6-407b-b185-139d66a36be2 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:41:03.339919 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5868b13f-66b3-49b6-b045-f7ad3b16430c admin admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:41:03.360502 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c522c7a-b9e3-48f5-9049-ed6b711dab04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:41:03.361196 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c522c7a-b9e3-48f5-9049-ed6b711dab04 admin admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:41:03.361568 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c522c7a-b9e3-48f5-9049-ed6b711dab04 admin admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:41:03.361856 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c522c7a-b9e3-48f5-9049-ed6b711dab04 admin admin] PATH_INFO: `/v3/roles` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:41:03.362654 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c522c7a-b9e3-48f5-9049-ed6b711dab04 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:41:03.364333 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c522c7a-b9e3-48f5-9049-ed6b711dab04 admin admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:41:03.420313 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 958/1900] 10.222.0.57 () {58 vars in 1121 bytes} [Fri Jul 1 16:41:03 2022] GET /identity/v3/roles => generated 1420 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 16:41:03.427168 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c522c7a-b9e3-48f5-9049-ed6b711dab04 admin admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:41:03.461798 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c522c7a-b9e3-48f5-9049-ed6b711dab04 admin admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:41:03.466335 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bfa266f-e682-4ac7-b263-772063c0b98b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:41:03.467035 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfa266f-e682-4ac7-b263-772063c0b98b admin admin] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:41:03.467322 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfa266f-e682-4ac7-b263-772063c0b98b admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:41:03.467604 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfa266f-e682-4ac7-b263-772063c0b98b admin admin] PATH_INFO: `/v3/projects` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:41:03.468315 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bfa266f-e682-4ac7-b263-772063c0b98b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:41:03.470318 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bfa266f-e682-4ac7-b263-772063c0b98b admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:41:03.489160 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f876f7-d4e5-452e-8ce0-ab66618b7f85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:41:03.490092 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f876f7-d4e5-452e-8ce0-ab66618b7f85 admin admin] REQUEST_METHOD: `PUT` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:41:03.490456 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f876f7-d4e5-452e-8ce0-ab66618b7f85 admin admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:41:03.490742 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f876f7-d4e5-452e-8ce0-ab66618b7f85 admin admin] PATH_INFO: `/v3/projects/10bf42ede27e4c9c8a2faa4dd348fbff/users/2f78e619ff21465ea0d4d6a19e640b6a/roles/84ee739146934c1ab32c501ff8093644` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:41:03.494625 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f876f7-d4e5-452e-8ce0-ab66618b7f85 admin admin] RBAC: Authorizing `identity:create_grant(project_id=10bf42ede27e4c9c8a2faa4dd348fbff, user_id=2f78e619ff21465ea0d4d6a19e640b6a, role_id=84ee739146934c1ab32c501ff8093644)` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:41:03.500037 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f876f7-d4e5-452e-8ce0-ab66618b7f85 admin admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:41:03.527680 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 959/1901] 10.222.0.57 () {60 vars in 1482 bytes} [Fri Jul 1 16:41:03 2022] PUT /identity/v3/projects/10bf42ede27e4c9c8a2faa4dd348fbff/users/2f78e619ff21465ea0d4d6a19e640b6a/roles/84ee739146934c1ab32c501ff8093644 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 16:41:03.540593 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b062f980-eb23-49b8-8a08-f8e3ea423416 None None] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:41:03.540886 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b062f980-eb23-49b8-8a08-f8e3ea423416 None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:41:03.541172 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b062f980-eb23-49b8-8a08-f8e3ea423416 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:41:03.582830 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 943/1902] 10.222.0.57 () {60 vars in 1152 bytes} [Fri Jul 1 16:41:03 2022] POST /identity/v3/projects => generated 355 bytes in 370 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 16:41:03.587984 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bfa266f-e682-4ac7-b263-772063c0b98b admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:41:03.607794 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.auth.core [None req-b062f980-eb23-49b8-8a08-f8e3ea423416 None None] MFA Rules not processed for user `2f78e619ff21465ea0d4d6a19e640b6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80312) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:41:03.609122 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-b062f980-eb23-49b8-8a08-f8e3ea423416 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:41:03.682968 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:41:03.775803 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bfa266f-e682-4ac7-b263-772063c0b98b admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:41:03.779292 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 960/1903] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 1 16:41:03 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 240 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 01 16:41:03.789274 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b062f980-eb23-49b8-8a08-f8e3ea423416 None None] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:41:03.791588 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-b062f980-eb23-49b8-8a08-f8e3ea423416 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:41:03.797991 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0a69a5-fb8a-4030-9a7f-4914e7e421d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:41:03.811743 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0a69a5-fb8a-4030-9a7f-4914e7e421d1 admin admin] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:41:03.813309 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0a69a5-fb8a-4030-9a7f-4914e7e421d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:41:03.816570 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0a69a5-fb8a-4030-9a7f-4914e7e421d1 admin admin] PATH_INFO: `/v3/users` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:41:03.818280 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a0a69a5-fb8a-4030-9a7f-4914e7e421d1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:41:03.820494 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a0a69a5-fb8a-4030-9a7f-4914e7e421d1 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:41:03.821723 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b062f980-eb23-49b8-8a08-f8e3ea423416 None None] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:41:03.834221 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660a4806-613c-44d9-946c-41d65a3bba3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:41:03.835517 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660a4806-613c-44d9-946c-41d65a3bba3c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:41:03.835885 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660a4806-613c-44d9-946c-41d65a3bba3c admin admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:41:03.836238 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660a4806-613c-44d9-946c-41d65a3bba3c admin admin] PATH_INFO: `/v3/users/2fefb68660aa47358a944f9f7840c10a` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:41:03.838921 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660a4806-613c-44d9-946c-41d65a3bba3c admin admin] RBAC: Authorizing `identity:delete_user(user_id=2fefb68660aa47358a944f9f7840c10a)` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:41:03.841435 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660a4806-613c-44d9-946c-41d65a3bba3c admin admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:41:03.887200 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 944/1904] 10.222.0.57 () {60 vars in 1143 bytes} [Fri Jul 1 16:41:03 2022] POST /identity/v3/users => generated 278 bytes in 300 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 16:41:03.888175 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0a69a5-fb8a-4030-9a7f-4914e7e421d1 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:41:03.890508 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-5a0a69a5-fb8a-4030-9a7f-4914e7e421d1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:41:03.931618 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-660a4806-613c-44d9-946c-41d65a3bba3c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2fefb68660aa47358a944f9f7840c10a'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:41:03.945957 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-660a4806-613c-44d9-946c-41d65a3bba3c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2fefb68660aa47358a944f9f7840c10a'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:41:03.956932 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-660a4806-613c-44d9-946c-41d65a3bba3c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2fefb68660aa47358a944f9f7840c10a'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:41:03.957880 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-660a4806-613c-44d9-946c-41d65a3bba3c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2fefb68660aa47358a944f9f7840c10a'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:41:03.958630 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-660a4806-613c-44d9-946c-41d65a3bba3c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2fefb68660aa47358a944f9f7840c10a'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:41:04.013957 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 961/1905] 10.222.0.57 () {58 vars in 1223 bytes} [Fri Jul 1 16:41:03 2022] DELETE /identity/v3/users/2fefb68660aa47358a944f9f7840c10a => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 16:41:04.025520 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660a4806-613c-44d9-946c-41d65a3bba3c admin admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:41:04.026970 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-660a4806-613c-44d9-946c-41d65a3bba3c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:41:04.044366 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0a69a5-fb8a-4030-9a7f-4914e7e421d1 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:41:04.061139 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:41:04.119009 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca41eca0-7670-4bb8-9642-20da0e3cd528 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbddf971a2354bbdbb9da1d63f3abf88', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0788e6725a84e57a4174284c1845755', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:41:04.119954 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca41eca0-7670-4bb8-9642-20da0e3cd528 service cinder] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:41:04.120236 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca41eca0-7670-4bb8-9642-20da0e3cd528 service cinder] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:41:04.120510 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca41eca0-7670-4bb8-9642-20da0e3cd528 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:41:04.123218 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-ca41eca0-7670-4bb8-9642-20da0e3cd528 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:41:04.165155 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660a4806-613c-44d9-946c-41d65a3bba3c admin admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:41:04.184286 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eea1184-5fe5-4bd7-9646-b2d7d85e256e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:41:04.185002 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eea1184-5fe5-4bd7-9646-b2d7d85e256e admin admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:41:04.185288 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eea1184-5fe5-4bd7-9646-b2d7d85e256e admin admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:41:04.185565 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eea1184-5fe5-4bd7-9646-b2d7d85e256e admin admin] PATH_INFO: `/v3/roles` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:41:04.186310 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eea1184-5fe5-4bd7-9646-b2d7d85e256e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:41:04.192670 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eea1184-5fe5-4bd7-9646-b2d7d85e256e admin admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:41:04.215657 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca41eca0-7670-4bb8-9642-20da0e3cd528 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:41:04.217286 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca41eca0-7670-4bb8-9642-20da0e3cd528 service cinder] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:41:04.220839 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 962/1906] 10.222.0.57 () {58 vars in 1121 bytes} [Fri Jul 1 16:41:04 2022] GET /identity/v3/roles => generated 1420 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 16:41:04.222592 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372067c0-9861-42b7-bdfc-764fbda1963e None None] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:41:04.222888 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372067c0-9861-42b7-bdfc-764fbda1963e None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:41:04.223170 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372067c0-9861-42b7-bdfc-764fbda1963e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:41:04.229619 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 945/1907] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 1 16:41:03 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 16:41:04.243421 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca41eca0-7670-4bb8-9642-20da0e3cd528 service cinder] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:41:04.245338 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.auth.core [None req-372067c0-9861-42b7-bdfc-764fbda1963e None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80312) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:41:04.246743 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-372067c0-9861-42b7-bdfc-764fbda1963e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:41:04.259929 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 963/1908] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 1 16:41:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 16:41:04.268442 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372067c0-9861-42b7-bdfc-764fbda1963e None None] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:41:04.270183 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-372067c0-9861-42b7-bdfc-764fbda1963e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:41:04.275893 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca41eca0-7670-4bb8-9642-20da0e3cd528 service cinder] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:41:04.288907 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec98ece-0384-43ad-b6ca-29b15876a89b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:41:04.289605 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec98ece-0384-43ad-b6ca-29b15876a89b admin admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:41:04.290095 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec98ece-0384-43ad-b6ca-29b15876a89b admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:41:04.290403 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec98ece-0384-43ad-b6ca-29b15876a89b admin admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:41:04.292641 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec98ece-0384-43ad-b6ca-29b15876a89b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:41:04.298601 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec98ece-0384-43ad-b6ca-29b15876a89b admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:41:04.327367 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 946/1909] 10.222.0.57 () {58 vars in 1121 bytes} [Fri Jul 1 16:41:04 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 16:41:04.332207 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec98ece-0384-43ad-b6ca-29b15876a89b admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:41:04.355104 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:41:04.362548 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec98ece-0384-43ad-b6ca-29b15876a89b admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:41:04.374637 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33288399-8b16-460e-b9d7-61b78bdc3cfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:41:04.375468 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33288399-8b16-460e-b9d7-61b78bdc3cfc admin admin] REQUEST_METHOD: `PUT` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:41:04.375782 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33288399-8b16-460e-b9d7-61b78bdc3cfc admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:41:04.376195 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33288399-8b16-460e-b9d7-61b78bdc3cfc admin admin] PATH_INFO: `/v3/projects/0776ab3ff39c4bbc8e4ba19eb6a978b9/users/9ebe4c339d454eeb861a7f5e1ca53d20/roles/983be40bdab04c478eaa90c6b6a46b14` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:41:04.391338 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372067c0-9861-42b7-bdfc-764fbda1963e None None] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:41:04.406960 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7580f19f-d40d-4559-8ca5-f320da713da0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ed200ce29684c5f80354fa9a8638fe2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0788e6725a84e57a4174284c1845755', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:41:04.407949 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7580f19f-d40d-4559-8ca5-f320da713da0 service neutron] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:41:04.409693 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7580f19f-d40d-4559-8ca5-f320da713da0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:41:04.410116 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7580f19f-d40d-4559-8ca5-f320da713da0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:41:04.411697 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33288399-8b16-460e-b9d7-61b78bdc3cfc admin admin] RBAC: Authorizing `identity:create_grant(project_id=0776ab3ff39c4bbc8e4ba19eb6a978b9, user_id=9ebe4c339d454eeb861a7f5e1ca53d20, role_id=983be40bdab04c478eaa90c6b6a46b14)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:41:04.415455 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33288399-8b16-460e-b9d7-61b78bdc3cfc admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:41:04.431440 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7580f19f-d40d-4559-8ca5-f320da713da0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:41:04.433167 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7580f19f-d40d-4559-8ca5-f320da713da0 service neutron] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:41:04.444458 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 947/1910] 10.222.0.57 () {60 vars in 1482 bytes} [Fri Jul 1 16:41:04 2022] PUT /identity/v3/projects/0776ab3ff39c4bbc8e4ba19eb6a978b9/users/9ebe4c339d454eeb861a7f5e1ca53d20/roles/983be40bdab04c478eaa90c6b6a46b14 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 16:41:04.455995 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33166413-e21e-448c-bd9f-69a42d902f4e None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:41:04.456307 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33166413-e21e-448c-bd9f-69a42d902f4e None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:41:04.456543 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33166413-e21e-448c-bd9f-69a42d902f4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:41:04.461766 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 964/1911] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 1 16:41:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 194 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 01 16:41:04.493254 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-33166413-e21e-448c-bd9f-69a42d902f4e None None] MFA Rules not processed for user `9ebe4c339d454eeb861a7f5e1ca53d20`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:41:04.494614 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-33166413-e21e-448c-bd9f-69a42d902f4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:41:04.581796 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 948/1912] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 1 16:41:04 2022] POST /identity/v3/auth/tokens => generated 3222 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 16:41:04.586825 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7580f19f-d40d-4559-8ca5-f320da713da0 service neutron] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:41:04.605412 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7580f19f-d40d-4559-8ca5-f320da713da0 service neutron] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:41:04.731403 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9cbaad5-7fc1-4da8-965a-2af151f8fa95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:41:04.735063 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9cbaad5-7fc1-4da8-965a-2af151f8fa95 admin admin] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:41:04.735360 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9cbaad5-7fc1-4da8-965a-2af151f8fa95 admin admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:41:04.735647 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9cbaad5-7fc1-4da8-965a-2af151f8fa95 admin admin] PATH_INFO: `/v3/projects` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:41:04.738188 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9cbaad5-7fc1-4da8-965a-2af151f8fa95 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:41:04.746423 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9cbaad5-7fc1-4da8-965a-2af151f8fa95 admin admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:41:04.805756 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 965/1913] 10.222.0.57 () {60 vars in 1152 bytes} [Fri Jul 1 16:41:04 2022] POST /identity/v3/projects => generated 355 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 16:41:04.813224 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33166413-e21e-448c-bd9f-69a42d902f4e None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:41:04.835175 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33166413-e21e-448c-bd9f-69a42d902f4e None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:41:04.837653 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d336b3-a258-48bf-a588-ea5807c1e403 None None] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:41:04.837895 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d336b3-a258-48bf-a588-ea5807c1e403 None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:41:04.838203 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d336b3-a258-48bf-a588-ea5807c1e403 None None] PATH_INFO: `/` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:41:04.839989 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 966/1914] 10.222.0.57 () {54 vars in 921 bytes} [Fri Jul 1 16:41:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 16:41:04.851752 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d336b3-a258-48bf-a588-ea5807c1e403 None None] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:41:04.853608 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-20d336b3-a258-48bf-a588-ea5807c1e403 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:41:04.871460 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:41:04.931529 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7042173c-fc0c-4c36-89e0-da421aace217 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:41:04.932224 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7042173c-fc0c-4c36-89e0-da421aace217 admin admin] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:41:04.932516 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7042173c-fc0c-4c36-89e0-da421aace217 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:41:04.932798 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7042173c-fc0c-4c36-89e0-da421aace217 admin admin] PATH_INFO: `/v3/users` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:41:04.933503 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7042173c-fc0c-4c36-89e0-da421aace217 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:41:04.936056 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7042173c-fc0c-4c36-89e0-da421aace217 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:41:04.973656 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 949/1915] 10.222.0.57 () {60 vars in 1143 bytes} [Fri Jul 1 16:41:04 2022] POST /identity/v3/users => generated 284 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 16:41:04.975632 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66afd56d-3155-43ff-ad10-ed07193940da None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:41:04.976198 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66afd56d-3155-43ff-ad10-ed07193940da None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:41:04.976628 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66afd56d-3155-43ff-ad10-ed07193940da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:41:04.999906 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-66afd56d-3155-43ff-ad10-ed07193940da None None] MFA Rules not processed for user `7a50f745b31941b7a7265c0fc852f48e`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:41:05.001411 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-66afd56d-3155-43ff-ad10-ed07193940da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:41:05.019097 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d336b3-a258-48bf-a588-ea5807c1e403 None None] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:41:05.026151 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c1fd27-8ba0-4663-908a-590bff8bddef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:41:05.027981 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c1fd27-8ba0-4663-908a-590bff8bddef admin admin] REQUEST_METHOD: `DELETE` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:41:05.028827 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c1fd27-8ba0-4663-908a-590bff8bddef admin admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:41:05.029099 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c1fd27-8ba0-4663-908a-590bff8bddef admin admin] PATH_INFO: `/v3/projects/89de13e71c8243e7a6f877cb0017b1a5` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:41:05.030988 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c1fd27-8ba0-4663-908a-590bff8bddef admin admin] RBAC: Authorizing `identity:delete_project(project_id=89de13e71c8243e7a6f877cb0017b1a5)` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:41:05.036155 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c1fd27-8ba0-4663-908a-590bff8bddef admin admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:41:05.104301 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-48c1fd27-8ba0-4663-908a-590bff8bddef admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '89de13e71c8243e7a6f877cb0017b1a5', 'request_id': 'req-48c1fd27-8ba0-4663-908a-590bff8bddef'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:41:05.119098 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-48c1fd27-8ba0-4663-908a-590bff8bddef admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '89de13e71c8243e7a6f877cb0017b1a5', 'request_id': 'req-48c1fd27-8ba0-4663-908a-590bff8bddef'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:41:05.119299 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 950/1916] 10.222.0.57 () {58 vars in 1018 bytes} [Fri Jul 1 16:41:04 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 16:41:05.123632 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=92,keystone:INSERT=2,keystone:DELETE=6 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:41:05.124018 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66afd56d-3155-43ff-ad10-ed07193940da None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:41:05.126767 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-48c1fd27-8ba0-4663-908a-590bff8bddef admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '89de13e71c8243e7a6f877cb0017b1a5', 'request_id': 'req-48c1fd27-8ba0-4663-908a-590bff8bddef'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:41:05.142013 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66afd56d-3155-43ff-ad10-ed07193940da None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:41:05.143108 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-48c1fd27-8ba0-4663-908a-590bff8bddef admin admin] The token cache is being invalidate because project 89de13e71c8243e7a6f877cb0017b1a5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80312) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 01 16:41:05.143417 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-48c1fd27-8ba0-4663-908a-590bff8bddef admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:41:05.144257 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-48c1fd27-8ba0-4663-908a-590bff8bddef admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:41:05.145994 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 967/1917] 10.222.0.57 () {58 vars in 1232 bytes} [Fri Jul 1 16:41:04 2022] DELETE /identity/v3/projects/89de13e71c8243e7a6f877cb0017b1a5 => generated 0 bytes in 295 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 01 16:41:05.149818 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c1fd27-8ba0-4663-908a-590bff8bddef admin admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:41:05.151206 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-48c1fd27-8ba0-4663-908a-590bff8bddef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:41:05.219273 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27dc0b6e-610b-41cc-ae29-29860df6a513 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:41:05.220138 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27dc0b6e-610b-41cc-ae29-29860df6a513 admin admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:41:05.220261 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27dc0b6e-610b-41cc-ae29-29860df6a513 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:41:05.220573 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27dc0b6e-610b-41cc-ae29-29860df6a513 admin admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:41:05.221708 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27dc0b6e-610b-41cc-ae29-29860df6a513 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:41:05.223628 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27dc0b6e-610b-41cc-ae29-29860df6a513 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:41:05.224297 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c1fd27-8ba0-4663-908a-590bff8bddef admin admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:41:05.229672 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a7e5c2-1b71-4d54-8347-dc59859fd469 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:41:05.230697 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a7e5c2-1b71-4d54-8347-dc59859fd469 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:41:05.231056 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a7e5c2-1b71-4d54-8347-dc59859fd469 admin admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:41:05.231407 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a7e5c2-1b71-4d54-8347-dc59859fd469 admin admin] PATH_INFO: `/v3/users/3a4041b57b9c4a1795bef1c741405414` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:41:05.232761 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a7e5c2-1b71-4d54-8347-dc59859fd469 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3a4041b57b9c4a1795bef1c741405414)` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:41:05.234886 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a7e5c2-1b71-4d54-8347-dc59859fd469 admin admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:41:05.239635 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 951/1918] 10.222.0.57 () {58 vars in 1121 bytes} [Fri Jul 1 16:41:05 2022] GET /identity/v3/roles => generated 1420 bytes in 116 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 16:41:05.243533 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27dc0b6e-610b-41cc-ae29-29860df6a513 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:41:05.246543 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-27dc0b6e-610b-41cc-ae29-29860df6a513 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:41:05.267005 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27dc0b6e-610b-41cc-ae29-29860df6a513 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:41:05.272924 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ddab5e9-aa77-4c63-a804-feeea8be67a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:41:05.273930 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddab5e9-aa77-4c63-a804-feeea8be67a7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:41:05.274282 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddab5e9-aa77-4c63-a804-feeea8be67a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:41:05.274605 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddab5e9-aa77-4c63-a804-feeea8be67a7 admin admin] PATH_INFO: `/v3/projects/b36385de9bfd4ba691c28ab5550aa7dc/users/38179860d6754ad4b1eb8d59cb2d16c6/roles/84ee739146934c1ab32c501ff8093644` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:41:05.297324 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-a4a7e5c2-1b71-4d54-8347-dc59859fd469 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3a4041b57b9c4a1795bef1c741405414'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:41:05.297591 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ddab5e9-aa77-4c63-a804-feeea8be67a7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b36385de9bfd4ba691c28ab5550aa7dc, user_id=38179860d6754ad4b1eb8d59cb2d16c6, role_id=84ee739146934c1ab32c501ff8093644)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:41:05.300761 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ddab5e9-aa77-4c63-a804-feeea8be67a7 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:41:05.312595 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-a4a7e5c2-1b71-4d54-8347-dc59859fd469 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3a4041b57b9c4a1795bef1c741405414'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:41:05.320874 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 952/1919] 10.222.0.57 () {60 vars in 1482 bytes} [Fri Jul 1 16:41:05 2022] PUT /identity/v3/projects/b36385de9bfd4ba691c28ab5550aa7dc/users/38179860d6754ad4b1eb8d59cb2d16c6/roles/84ee739146934c1ab32c501ff8093644 => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 16:41:05.321992 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ddab5e9-aa77-4c63-a804-feeea8be67a7 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:41:05.323588 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-a4a7e5c2-1b71-4d54-8347-dc59859fd469 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3a4041b57b9c4a1795bef1c741405414'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:41:05.324422 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-a4a7e5c2-1b71-4d54-8347-dc59859fd469 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3a4041b57b9c4a1795bef1c741405414'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:41:05.325169 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-a4a7e5c2-1b71-4d54-8347-dc59859fd469 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3a4041b57b9c4a1795bef1c741405414'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:41:05.338393 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:41:05.342157 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 968/1920] 10.222.0.57 () {58 vars in 1223 bytes} [Fri Jul 1 16:41:05 2022] DELETE /identity/v3/users/3a4041b57b9c4a1795bef1c741405414 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 01 16:41:05.403103 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ddab5e9-aa77-4c63-a804-feeea8be67a7 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:41:05.486264 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6370eb-0f5c-4893-83eb-ddc583e93409 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:41:05.486565 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6370eb-0f5c-4893-83eb-ddc583e93409 admin admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:41:05.487060 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6370eb-0f5c-4893-83eb-ddc583e93409 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:41:05.487405 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6370eb-0f5c-4893-83eb-ddc583e93409 admin admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:41:05.488248 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc6370eb-0f5c-4893-83eb-ddc583e93409 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:41:05.489989 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc6370eb-0f5c-4893-83eb-ddc583e93409 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:41:05.506520 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 953/1921] 10.222.0.57 () {58 vars in 1121 bytes} [Fri Jul 1 16:41:05 2022] GET /identity/v3/roles => generated 1420 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 01 16:41:05.507657 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a7e5c2-1b71-4d54-8347-dc59859fd469 admin admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:41:05.509290 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-a4a7e5c2-1b71-4d54-8347-dc59859fd469 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:41:05.552328 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a7e5c2-1b71-4d54-8347-dc59859fd469 admin admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:41:05.558275 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6370eb-0f5c-4893-83eb-ddc583e93409 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:41:05.560320 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-bc6370eb-0f5c-4893-83eb-ddc583e93409 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:41:05.563044 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f947377b-9519-4323-ad00-73f1cf1ea459 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:41:05.563963 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f947377b-9519-4323-ad00-73f1cf1ea459 admin admin] REQUEST_METHOD: `PUT` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:41:05.564257 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f947377b-9519-4323-ad00-73f1cf1ea459 admin admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:41:05.565066 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f947377b-9519-4323-ad00-73f1cf1ea459 admin admin] PATH_INFO: `/v3/projects/b36385de9bfd4ba691c28ab5550aa7dc/users/38179860d6754ad4b1eb8d59cb2d16c6/roles/84ee739146934c1ab32c501ff8093644` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:41:05.569416 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f947377b-9519-4323-ad00-73f1cf1ea459 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b36385de9bfd4ba691c28ab5550aa7dc, user_id=38179860d6754ad4b1eb8d59cb2d16c6, role_id=84ee739146934c1ab32c501ff8093644)` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:41:05.572360 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f947377b-9519-4323-ad00-73f1cf1ea459 admin admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:41:05.588025 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 969/1922] 10.222.0.57 () {60 vars in 1482 bytes} [Fri Jul 1 16:41:05 2022] PUT /identity/v3/projects/b36385de9bfd4ba691c28ab5550aa7dc/users/38179860d6754ad4b1eb8d59cb2d16c6/roles/84ee739146934c1ab32c501ff8093644 => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 01 16:41:05.603665 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b2e7d8-3111-43a7-9484-6431fc7249e1 None None] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:41:05.604263 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b2e7d8-3111-43a7-9484-6431fc7249e1 None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:41:05.605832 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b2e7d8-3111-43a7-9484-6431fc7249e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:41:05.648831 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.auth.core [None req-29b2e7d8-3111-43a7-9484-6431fc7249e1 None None] MFA Rules not processed for user `38179860d6754ad4b1eb8d59cb2d16c6`. Rule list: `[]` (Enabled: `True`). {{(pid=80312) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:41:05.650432 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-29b2e7d8-3111-43a7-9484-6431fc7249e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:41:05.676261 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6370eb-0f5c-4893-83eb-ddc583e93409 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:41:05.739551 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:DELETE=11,keystone:SELECT=87,keystone:INSERT=2 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:41:05.756218 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:41:05.787442 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 970/1923] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 1 16:41:05 2022] POST /identity/v3/auth/tokens => generated 3289 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 16:41:05.787897 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79a407c-9fca-466b-b8c6-eb2e0796566d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a50f745b31941b7a7265c0fc852f48e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0788e6725a84e57a4174284c1845755', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:41:05.788354 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b2e7d8-3111-43a7-9484-6431fc7249e1 None None] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:41:05.789141 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79a407c-9fca-466b-b8c6-eb2e0796566d service nova] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:41:05.789573 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79a407c-9fca-466b-b8c6-eb2e0796566d service nova] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:41:05.789694 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-29b2e7d8-3111-43a7-9484-6431fc7249e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:41:05.790289 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79a407c-9fca-466b-b8c6-eb2e0796566d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:41:05.793094 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-f79a407c-9fca-466b-b8c6-eb2e0796566d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:41:05.822040 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f79a407c-9fca-466b-b8c6-eb2e0796566d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:41:05.824663 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f79a407c-9fca-466b-b8c6-eb2e0796566d service nova] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:41:05.841170 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 954/1924] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 1 16:41:05 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 283 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 16:41:05.842050 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79a407c-9fca-466b-b8c6-eb2e0796566d service nova] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:41:05.849267 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-f79a407c-9fca-466b-b8c6-eb2e0796566d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:41:05.925969 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b2e7d8-3111-43a7-9484-6431fc7249e1 None None] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:41:05.946478 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1230bf1f-3201-4cf7-ac61-f31bb26707d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:41:05.946478 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1230bf1f-3201-4cf7-ac61-f31bb26707d8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:41:05.946478 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1230bf1f-3201-4cf7-ac61-f31bb26707d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:41:05.946478 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1230bf1f-3201-4cf7-ac61-f31bb26707d8 admin admin] PATH_INFO: `/v3/projects/0ea969a45167484fb43be6f922feec15` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:41:05.949209 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1230bf1f-3201-4cf7-ac61-f31bb26707d8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0ea969a45167484fb43be6f922feec15)` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:41:05.952908 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1230bf1f-3201-4cf7-ac61-f31bb26707d8 admin admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:41:05.954102 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79a407c-9fca-466b-b8c6-eb2e0796566d service nova] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:41:05.958828 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fdb7d7c-205b-40e0-b19e-70d8690fa3b3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbddf971a2354bbdbb9da1d63f3abf88', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0788e6725a84e57a4174284c1845755', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:41:05.960340 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdb7d7c-205b-40e0-b19e-70d8690fa3b3 service cinder] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:41:05.960340 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdb7d7c-205b-40e0-b19e-70d8690fa3b3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:41:05.960578 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdb7d7c-205b-40e0-b19e-70d8690fa3b3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:41:05.972347 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fdb7d7c-205b-40e0-b19e-70d8690fa3b3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:41:05.974090 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fdb7d7c-205b-40e0-b19e-70d8690fa3b3 service cinder] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:41:05.985507 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 955/1925] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 1 16:41:05 2022] GET /identity/v3/auth/tokens => generated 3289 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 16:41:06.096600 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-1230bf1f-3201-4cf7-ac61-f31bb26707d8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0ea969a45167484fb43be6f922feec15', 'request_id': 'req-1230bf1f-3201-4cf7-ac61-f31bb26707d8'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:41:06.117117 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-1230bf1f-3201-4cf7-ac61-f31bb26707d8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0ea969a45167484fb43be6f922feec15', 'request_id': 'req-1230bf1f-3201-4cf7-ac61-f31bb26707d8'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:41:06.129502 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-1230bf1f-3201-4cf7-ac61-f31bb26707d8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0ea969a45167484fb43be6f922feec15', 'request_id': 'req-1230bf1f-3201-4cf7-ac61-f31bb26707d8'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:41:06.156015 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-1230bf1f-3201-4cf7-ac61-f31bb26707d8 admin admin] The token cache is being invalidate because project 0ea969a45167484fb43be6f922feec15 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80312) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 01 16:41:06.156403 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-1230bf1f-3201-4cf7-ac61-f31bb26707d8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:41:06.157669 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-1230bf1f-3201-4cf7-ac61-f31bb26707d8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:41:06.162527 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 971/1926] 10.222.0.57 () {58 vars in 1232 bytes} [Fri Jul 1 16:41:05 2022] DELETE /identity/v3/projects/0ea969a45167484fb43be6f922feec15 => generated 0 bytes in 372 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 16:41:06.170417 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fb5f95-5d96-4220-8917-391d4a3f5d2b None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:41:06.170921 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fb5f95-5d96-4220-8917-391d4a3f5d2b None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:41:06.171109 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fb5f95-5d96-4220-8917-391d4a3f5d2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:41:06.194584 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1230bf1f-3201-4cf7-ac61-f31bb26707d8 admin admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:41:06.197239 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-1230bf1f-3201-4cf7-ac61-f31bb26707d8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:41:06.200262 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-d2fb5f95-5d96-4220-8917-391d4a3f5d2b None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:41:06.201575 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-d2fb5f95-5d96-4220-8917-391d4a3f5d2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:41:06.297662 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 956/1927] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 1 16:41:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 16:41:06.299596 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2fb5f95-5d96-4220-8917-391d4a3f5d2b None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:41:06.301112 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-d2fb5f95-5d96-4220-8917-391d4a3f5d2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:41:06.357755 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1230bf1f-3201-4cf7-ac61-f31bb26707d8 admin admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:41:06.369802 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a558f09d-9280-4dff-bf14-08aacca246c5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbddf971a2354bbdbb9da1d63f3abf88', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0788e6725a84e57a4174284c1845755', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:41:06.370573 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a558f09d-9280-4dff-bf14-08aacca246c5 service cinder] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:41:06.370896 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a558f09d-9280-4dff-bf14-08aacca246c5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:41:06.372293 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a558f09d-9280-4dff-bf14-08aacca246c5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:41:06.376144 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2fb5f95-5d96-4220-8917-391d4a3f5d2b None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:41:06.378393 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-a558f09d-9280-4dff-bf14-08aacca246c5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:41:06.384995 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9497108-c5ab-4c7b-9fdd-233e147e3c90 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbddf971a2354bbdbb9da1d63f3abf88', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0788e6725a84e57a4174284c1845755', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:41:06.385835 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9497108-c5ab-4c7b-9fdd-233e147e3c90 service cinder] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:41:06.386526 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9497108-c5ab-4c7b-9fdd-233e147e3c90 service cinder] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:41:06.387014 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9497108-c5ab-4c7b-9fdd-233e147e3c90 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:41:06.389964 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-f9497108-c5ab-4c7b-9fdd-233e147e3c90 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:41:06.441616 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:41:06.481530 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:41:06.520289 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a558f09d-9280-4dff-bf14-08aacca246c5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:41:06.522997 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9497108-c5ab-4c7b-9fdd-233e147e3c90 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:41:06.523166 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a558f09d-9280-4dff-bf14-08aacca246c5 service cinder] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:41:06.525327 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9497108-c5ab-4c7b-9fdd-233e147e3c90 service cinder] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:41:06.540153 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 957/1928] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 1 16:41:06 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 242 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 16:41:06.542005 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fed6802-6c13-4bde-9111-fd1bdc6b3c66 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:41:06.542779 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fed6802-6c13-4bde-9111-fd1bdc6b3c66 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:41:06.543287 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fed6802-6c13-4bde-9111-fd1bdc6b3c66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:41:06.544301 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 972/1929] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 1 16:41:06 2022] GET /identity/v3/auth/tokens => generated 3222 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 01 16:41:06.575845 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-1fed6802-6c13-4bde-9111-fd1bdc6b3c66 None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:41:06.577484 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-1fed6802-6c13-4bde-9111-fd1bdc6b3c66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:41:06.592481 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 958/1930] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 1 16:41:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 16:41:06.593200 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a558f09d-9280-4dff-bf14-08aacca246c5 service cinder] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:41:06.612446 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a558f09d-9280-4dff-bf14-08aacca246c5 service cinder] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:41:06.621978 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3fe5f4-6578-4c2c-a271-c63b83ea7945 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:41:06.622851 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3fe5f4-6578-4c2c-a271-c63b83ea7945 admin admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:41:06.623350 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3fe5f4-6578-4c2c-a271-c63b83ea7945 admin admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:41:06.623650 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3fe5f4-6578-4c2c-a271-c63b83ea7945 admin admin] PATH_INFO: `/v3/domains` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:41:06.624409 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb3fe5f4-6578-4c2c-a271-c63b83ea7945 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80312) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:41:06.624724 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb3fe5f4-6578-4c2c-a271-c63b83ea7945 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:41:06.629594 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb3fe5f4-6578-4c2c-a271-c63b83ea7945 admin admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:41:06.649881 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 973/1931] 10.222.0.57 () {58 vars in 1152 bytes} [Fri Jul 1 16:41:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:41:06.651188 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fed6802-6c13-4bde-9111-fd1bdc6b3c66 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:41:06.663827 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fed6802-6c13-4bde-9111-fd1bdc6b3c66 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:41:06.673264 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ede9a6-f085-49e6-95f4-7d5d959340d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:41:06.675626 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ede9a6-f085-49e6-95f4-7d5d959340d2 admin admin] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:41:06.675626 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ede9a6-f085-49e6-95f4-7d5d959340d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:41:06.675928 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ede9a6-f085-49e6-95f4-7d5d959340d2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:41:06.677538 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ede9a6-f085-49e6-95f4-7d5d959340d2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:41:06.680554 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ede9a6-f085-49e6-95f4-7d5d959340d2 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:41:06.730233 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 959/1932] 10.222.0.57 () {60 vars in 1152 bytes} [Fri Jul 1 16:41:06 2022] POST /identity/v3/projects => generated 385 bytes in 79 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 16:41:06.733868 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3fe5f4-6578-4c2c-a271-c63b83ea7945 admin admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:41:06.747848 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3fe5f4-6578-4c2c-a271-c63b83ea7945 admin admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:41:06.858016 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26317252-0362-475b-be9a-10f0453024ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:41:06.858016 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26317252-0362-475b-be9a-10f0453024ec admin admin] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:41:06.858318 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26317252-0362-475b-be9a-10f0453024ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:41:06.858750 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26317252-0362-475b-be9a-10f0453024ec admin admin] PATH_INFO: `/v3/users` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:41:06.860848 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26317252-0362-475b-be9a-10f0453024ec admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:41:06.865876 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26317252-0362-475b-be9a-10f0453024ec admin admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:41:06.896376 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 974/1933] 10.222.0.57 () {60 vars in 1143 bytes} [Fri Jul 1 16:41:06 2022] POST /identity/v3/users => generated 293 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 16:41:06.900227 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ede9a6-f085-49e6-95f4-7d5d959340d2 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:41:06.912802 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ede9a6-f085-49e6-95f4-7d5d959340d2 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:41:06.920293 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8c7a3f-cb49-4460-9139-900848788211 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:41:06.921201 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8c7a3f-cb49-4460-9139-900848788211 admin admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:41:06.921668 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8c7a3f-cb49-4460-9139-900848788211 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:41:06.922195 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8c7a3f-cb49-4460-9139-900848788211 admin admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:41:06.922894 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8c7a3f-cb49-4460-9139-900848788211 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:41:06.927340 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8c7a3f-cb49-4460-9139-900848788211 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:41:06.955912 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 960/1934] 10.222.0.57 () {58 vars in 1121 bytes} [Fri Jul 1 16:41:06 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 16:41:06.961049 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26317252-0362-475b-be9a-10f0453024ec admin admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:41:06.978872 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26317252-0362-475b-be9a-10f0453024ec admin admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:41:06.999251 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c20154-cee3-452e-a4e3-adbf5a2250d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:41:07.001450 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c20154-cee3-452e-a4e3-adbf5a2250d1 admin admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:41:07.003160 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c20154-cee3-452e-a4e3-adbf5a2250d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:41:07.003880 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c20154-cee3-452e-a4e3-adbf5a2250d1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:41:07.005008 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c20154-cee3-452e-a4e3-adbf5a2250d1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:41:07.006979 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c20154-cee3-452e-a4e3-adbf5a2250d1 admin admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:41:07.024894 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 975/1935] 10.222.0.57 () {58 vars in 1121 bytes} [Fri Jul 1 16:41:06 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 01 16:41:07.025878 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8c7a3f-cb49-4460-9139-900848788211 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:41:07.050475 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8c7a3f-cb49-4460-9139-900848788211 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:41:07.059898 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2c9dc1-3e58-4802-86e1-c48dca359da6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:41:07.061801 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2c9dc1-3e58-4802-86e1-c48dca359da6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:41:07.064208 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2c9dc1-3e58-4802-86e1-c48dca359da6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:41:07.065592 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2c9dc1-3e58-4802-86e1-c48dca359da6 admin admin] PATH_INFO: `/v3/projects/c1f4b14fffe64622a01f28a35fc94f3a/users/98bab471ebe042619ab5f1ba744d58e9/roles/983be40bdab04c478eaa90c6b6a46b14` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:41:07.092884 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa2c9dc1-3e58-4802-86e1-c48dca359da6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c1f4b14fffe64622a01f28a35fc94f3a, user_id=98bab471ebe042619ab5f1ba744d58e9, role_id=983be40bdab04c478eaa90c6b6a46b14)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:41:07.097404 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa2c9dc1-3e58-4802-86e1-c48dca359da6 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:41:07.127792 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 961/1936] 10.222.0.57 () {60 vars in 1482 bytes} [Fri Jul 1 16:41:07 2022] PUT /identity/v3/projects/c1f4b14fffe64622a01f28a35fc94f3a/users/98bab471ebe042619ab5f1ba744d58e9/roles/983be40bdab04c478eaa90c6b6a46b14 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 16:41:07.138614 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2bbe73-ae0a-4f8a-9ff3-21fb0a12fabb None None] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:41:07.138957 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2bbe73-ae0a-4f8a-9ff3-21fb0a12fabb None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:41:07.139254 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2bbe73-ae0a-4f8a-9ff3-21fb0a12fabb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:41:07.178466 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.auth.core [None req-4b2bbe73-ae0a-4f8a-9ff3-21fb0a12fabb None None] MFA Rules not processed for user `98bab471ebe042619ab5f1ba744d58e9`. Rule list: `[]` (Enabled: `True`). {{(pid=80312) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:41:07.180533 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-4b2bbe73-ae0a-4f8a-9ff3-21fb0a12fabb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:41:07.265978 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 976/1937] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 1 16:41:07 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 16:41:07.271529 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2c9dc1-3e58-4802-86e1-c48dca359da6 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:41:07.372187 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2c9dc1-3e58-4802-86e1-c48dca359da6 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:41:07.379535 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a4f43d-424b-49c5-80af-3b7babb38e84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:41:07.380343 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a4f43d-424b-49c5-80af-3b7babb38e84 admin admin] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:41:07.380637 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a4f43d-424b-49c5-80af-3b7babb38e84 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:41:07.380919 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a4f43d-424b-49c5-80af-3b7babb38e84 admin admin] PATH_INFO: `/v3/projects` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:41:07.381668 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a4f43d-424b-49c5-80af-3b7babb38e84 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:41:07.384305 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a4f43d-424b-49c5-80af-3b7babb38e84 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:41:07.420769 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 962/1938] 10.222.0.57 () {60 vars in 1152 bytes} [Fri Jul 1 16:41:07 2022] POST /identity/v3/projects => generated 385 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 16:41:07.427411 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2bbe73-ae0a-4f8a-9ff3-21fb0a12fabb None None] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:41:07.442871 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2bbe73-ae0a-4f8a-9ff3-21fb0a12fabb None None] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:41:07.469853 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:41:07.551346 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238e5a56-ac15-4a1f-9b0a-a3f65b4879f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:41:07.553071 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238e5a56-ac15-4a1f-9b0a-a3f65b4879f4 admin admin] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:41:07.553366 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238e5a56-ac15-4a1f-9b0a-a3f65b4879f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:41:07.553651 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238e5a56-ac15-4a1f-9b0a-a3f65b4879f4 admin admin] PATH_INFO: `/v3/users` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:41:07.554478 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238e5a56-ac15-4a1f-9b0a-a3f65b4879f4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:41:07.556874 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238e5a56-ac15-4a1f-9b0a-a3f65b4879f4 admin admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:41:07.583813 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a4f43d-424b-49c5-80af-3b7babb38e84 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:41:07.588424 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 977/1939] 10.222.0.57 () {60 vars in 1143 bytes} [Fri Jul 1 16:41:07 2022] POST /identity/v3/users => generated 299 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 16:41:07.589502 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238e5a56-ac15-4a1f-9b0a-a3f65b4879f4 admin admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:41:07.601643 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238e5a56-ac15-4a1f-9b0a-a3f65b4879f4 admin admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:41:07.606635 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16628a76-06f6-40d0-a4f3-2ff81c278da6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:41:07.607315 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16628a76-06f6-40d0-a4f3-2ff81c278da6 admin admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:41:07.607605 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16628a76-06f6-40d0-a4f3-2ff81c278da6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:41:07.608027 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16628a76-06f6-40d0-a4f3-2ff81c278da6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:41:07.608776 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16628a76-06f6-40d0-a4f3-2ff81c278da6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:41:07.610565 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16628a76-06f6-40d0-a4f3-2ff81c278da6 admin admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:41:07.625323 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 978/1940] 10.222.0.57 () {58 vars in 1121 bytes} [Fri Jul 1 16:41:07 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 01 16:41:07.626377 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16628a76-06f6-40d0-a4f3-2ff81c278da6 admin admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:41:07.637836 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16628a76-06f6-40d0-a4f3-2ff81c278da6 admin admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:41:07.642930 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde63d5a-9303-4ead-a7bf-7a6aa45ca5e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:41:07.643817 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde63d5a-9303-4ead-a7bf-7a6aa45ca5e4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:41:07.644109 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde63d5a-9303-4ead-a7bf-7a6aa45ca5e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:41:07.644389 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde63d5a-9303-4ead-a7bf-7a6aa45ca5e4 admin admin] PATH_INFO: `/v3/projects/023fbbb1d8354bf39a57448345b1c59b/users/165833ceb4c84f76bd128b726d0e31d9/roles/84ee739146934c1ab32c501ff8093644` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:41:07.658409 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a4f43d-424b-49c5-80af-3b7babb38e84 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:41:07.661261 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde63d5a-9303-4ead-a7bf-7a6aa45ca5e4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=023fbbb1d8354bf39a57448345b1c59b, user_id=165833ceb4c84f76bd128b726d0e31d9, role_id=84ee739146934c1ab32c501ff8093644)` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:41:07.662247 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21fa97db-b366-471a-96db-718422ef679c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbddf971a2354bbdbb9da1d63f3abf88', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0788e6725a84e57a4174284c1845755', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:41:07.662967 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fa97db-b366-471a-96db-718422ef679c service cinder] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:41:07.663258 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fa97db-b366-471a-96db-718422ef679c service cinder] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:41:07.663554 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fa97db-b366-471a-96db-718422ef679c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:41:07.664081 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde63d5a-9303-4ead-a7bf-7a6aa45ca5e4 admin admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:41:07.665295 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-21fa97db-b366-471a-96db-718422ef679c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:41:07.678995 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:41:07.682320 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 979/1941] 10.222.0.57 () {60 vars in 1482 bytes} [Fri Jul 1 16:41:07 2022] PUT /identity/v3/projects/023fbbb1d8354bf39a57448345b1c59b/users/165833ceb4c84f76bd128b726d0e31d9/roles/84ee739146934c1ab32c501ff8093644 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 16:41:07.683136 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde63d5a-9303-4ead-a7bf-7a6aa45ca5e4 admin admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:41:07.733141 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21fa97db-b366-471a-96db-718422ef679c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:41:07.734611 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21fa97db-b366-471a-96db-718422ef679c service cinder] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:41:07.744253 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 963/1942] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 1 16:41:07 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 01 16:41:07.747952 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde63d5a-9303-4ead-a7bf-7a6aa45ca5e4 admin admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:41:07.753765 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d021403-7e01-4a31-b4d3-d2fd23ded6ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:41:07.754408 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d021403-7e01-4a31-b4d3-d2fd23ded6ab admin admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:41:07.754702 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d021403-7e01-4a31-b4d3-d2fd23ded6ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:41:07.754978 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d021403-7e01-4a31-b4d3-d2fd23ded6ab admin admin] PATH_INFO: `/v3/roles` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:41:07.755711 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d021403-7e01-4a31-b4d3-d2fd23ded6ab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:41:07.757316 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d021403-7e01-4a31-b4d3-d2fd23ded6ab admin admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:41:07.768463 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 980/1943] 10.222.0.57 () {58 vars in 1121 bytes} [Fri Jul 1 16:41:07 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 01 16:41:07.772691 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21fa97db-b366-471a-96db-718422ef679c service cinder] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:41:07.783157 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21fa97db-b366-471a-96db-718422ef679c service cinder] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:41:07.788430 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552b9085-f01c-4326-90a5-e2551362f6f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:41:07.789325 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552b9085-f01c-4326-90a5-e2551362f6f9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:41:07.789610 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552b9085-f01c-4326-90a5-e2551362f6f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:41:07.789891 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552b9085-f01c-4326-90a5-e2551362f6f9 admin admin] PATH_INFO: `/v3/projects/023fbbb1d8354bf39a57448345b1c59b/users/165833ceb4c84f76bd128b726d0e31d9/roles/84ee739146934c1ab32c501ff8093644` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:41:07.791904 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-552b9085-f01c-4326-90a5-e2551362f6f9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=023fbbb1d8354bf39a57448345b1c59b, user_id=165833ceb4c84f76bd128b726d0e31d9, role_id=84ee739146934c1ab32c501ff8093644)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:41:07.794791 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-552b9085-f01c-4326-90a5-e2551362f6f9 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:41:07.805603 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 964/1944] 10.222.0.57 () {60 vars in 1482 bytes} [Fri Jul 1 16:41:07 2022] PUT /identity/v3/projects/023fbbb1d8354bf39a57448345b1c59b/users/165833ceb4c84f76bd128b726d0e31d9/roles/84ee739146934c1ab32c501ff8093644 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 01 16:41:07.817342 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44685c1c-1b8f-4142-bdcb-bc7fb0f0de3d None None] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:41:07.817701 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44685c1c-1b8f-4142-bdcb-bc7fb0f0de3d None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:41:07.818050 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44685c1c-1b8f-4142-bdcb-bc7fb0f0de3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:41:07.847887 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.auth.core [None req-44685c1c-1b8f-4142-bdcb-bc7fb0f0de3d None None] MFA Rules not processed for user `165833ceb4c84f76bd128b726d0e31d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80312) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:41:07.850106 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-44685c1c-1b8f-4142-bdcb-bc7fb0f0de3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:41:07.910309 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:41:07.920334 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 981/1945] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 1 16:41:07 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 16:41:07.931647 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552b9085-f01c-4326-90a5-e2551362f6f9 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:41:07.989550 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552b9085-f01c-4326-90a5-e2551362f6f9 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:41:07.993205 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944a6b64-bb10-4259-8933-ba2fc32c48a7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbddf971a2354bbdbb9da1d63f3abf88', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0788e6725a84e57a4174284c1845755', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:41:07.993845 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944a6b64-bb10-4259-8933-ba2fc32c48a7 service cinder] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:41:07.994131 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944a6b64-bb10-4259-8933-ba2fc32c48a7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:41:07.994439 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944a6b64-bb10-4259-8933-ba2fc32c48a7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:41:08.004505 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-944a6b64-bb10-4259-8933-ba2fc32c48a7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:41:08.005868 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-944a6b64-bb10-4259-8933-ba2fc32c48a7 service cinder] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:41:08.015440 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 965/1946] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 1 16:41:07 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 16:41:08.044295 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44685c1c-1b8f-4142-bdcb-bc7fb0f0de3d None None] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:41:08.115592 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44685c1c-1b8f-4142-bdcb-bc7fb0f0de3d None None] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:41:08.124207 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe84c06-358e-4d15-b25b-4f54e597d885 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:41:08.125261 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe84c06-358e-4d15-b25b-4f54e597d885 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:41:08.125561 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe84c06-358e-4d15-b25b-4f54e597d885 admin admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:41:08.125847 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe84c06-358e-4d15-b25b-4f54e597d885 admin admin] PATH_INFO: `/v3/users/98bab471ebe042619ab5f1ba744d58e9` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:41:08.128430 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe84c06-358e-4d15-b25b-4f54e597d885 admin admin] RBAC: Authorizing `identity:delete_user(user_id=98bab471ebe042619ab5f1ba744d58e9)` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:41:08.129855 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe84c06-358e-4d15-b25b-4f54e597d885 admin admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:41:08.183814 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-dbe84c06-358e-4d15-b25b-4f54e597d885 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '98bab471ebe042619ab5f1ba744d58e9'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:41:08.197093 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-dbe84c06-358e-4d15-b25b-4f54e597d885 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '98bab471ebe042619ab5f1ba744d58e9'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:41:08.204194 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-dbe84c06-358e-4d15-b25b-4f54e597d885 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '98bab471ebe042619ab5f1ba744d58e9'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:41:08.204954 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-dbe84c06-358e-4d15-b25b-4f54e597d885 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '98bab471ebe042619ab5f1ba744d58e9'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:41:08.206237 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-dbe84c06-358e-4d15-b25b-4f54e597d885 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '98bab471ebe042619ab5f1ba744d58e9'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:41:08.227819 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 982/1947] 10.222.0.57 () {58 vars in 1223 bytes} [Fri Jul 1 16:41:08 2022] DELETE /identity/v3/users/98bab471ebe042619ab5f1ba744d58e9 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 16:41:08.232370 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c534ee2d-04a1-4a18-b8d4-f276103f9d5a None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:41:08.232663 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c534ee2d-04a1-4a18-b8d4-f276103f9d5a None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:41:08.232949 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c534ee2d-04a1-4a18-b8d4-f276103f9d5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:41:08.251260 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-c534ee2d-04a1-4a18-b8d4-f276103f9d5a None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:41:08.252691 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-c534ee2d-04a1-4a18-b8d4-f276103f9d5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:41:08.312631 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 966/1948] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 1 16:41:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 01 16:41:08.322894 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe84c06-358e-4d15-b25b-4f54e597d885 admin admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:41:08.324372 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-dbe84c06-358e-4d15-b25b-4f54e597d885 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:41:08.384898 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe84c06-358e-4d15-b25b-4f54e597d885 admin admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:41:08.388517 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32811c1-40b0-48e7-a80d-9b5a28f6765a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ed200ce29684c5f80354fa9a8638fe2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0788e6725a84e57a4174284c1845755', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:41:08.389171 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32811c1-40b0-48e7-a80d-9b5a28f6765a service neutron] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:41:08.389457 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32811c1-40b0-48e7-a80d-9b5a28f6765a service neutron] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:41:08.389737 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32811c1-40b0-48e7-a80d-9b5a28f6765a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:41:08.400670 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c32811c1-40b0-48e7-a80d-9b5a28f6765a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:41:08.402166 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c32811c1-40b0-48e7-a80d-9b5a28f6765a service neutron] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:41:08.413202 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 983/1949] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 1 16:41:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 16:41:08.742333 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c534ee2d-04a1-4a18-b8d4-f276103f9d5a None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:41:08.744679 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-c534ee2d-04a1-4a18-b8d4-f276103f9d5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:41:08.760242 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c534ee2d-04a1-4a18-b8d4-f276103f9d5a None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:41:08.772017 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4cae08-1b3b-4536-ae4f-2dfafdd61a55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:41:08.772834 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4cae08-1b3b-4536-ae4f-2dfafdd61a55 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:41:08.773126 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4cae08-1b3b-4536-ae4f-2dfafdd61a55 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:41:08.773437 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4cae08-1b3b-4536-ae4f-2dfafdd61a55 admin admin] PATH_INFO: `/v3/projects/c1f4b14fffe64622a01f28a35fc94f3a` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:41:08.775048 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae4cae08-1b3b-4536-ae4f-2dfafdd61a55 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c1f4b14fffe64622a01f28a35fc94f3a)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:41:08.777193 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae4cae08-1b3b-4536-ae4f-2dfafdd61a55 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:41:08.827489 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=97,keystone:INSERT=1,keystone:DELETE=2 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:41:08.847881 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-ae4cae08-1b3b-4536-ae4f-2dfafdd61a55 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c1f4b14fffe64622a01f28a35fc94f3a', 'request_id': 'req-ae4cae08-1b3b-4536-ae4f-2dfafdd61a55'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:41:08.858227 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-ae4cae08-1b3b-4536-ae4f-2dfafdd61a55 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c1f4b14fffe64622a01f28a35fc94f3a', 'request_id': 'req-ae4cae08-1b3b-4536-ae4f-2dfafdd61a55'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:41:08.867113 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-ae4cae08-1b3b-4536-ae4f-2dfafdd61a55 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c1f4b14fffe64622a01f28a35fc94f3a', 'request_id': 'req-ae4cae08-1b3b-4536-ae4f-2dfafdd61a55'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:41:08.881809 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-ae4cae08-1b3b-4536-ae4f-2dfafdd61a55 admin admin] The token cache is being invalidate because project c1f4b14fffe64622a01f28a35fc94f3a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80311) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 01 16:41:08.882289 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-ae4cae08-1b3b-4536-ae4f-2dfafdd61a55 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:41:08.883186 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-ae4cae08-1b3b-4536-ae4f-2dfafdd61a55 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:41:08.888661 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 967/1950] 10.222.0.57 () {58 vars in 1232 bytes} [Fri Jul 1 16:41:08 2022] DELETE /identity/v3/projects/c1f4b14fffe64622a01f28a35fc94f3a => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 16:41:08.889887 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32811c1-40b0-48e7-a80d-9b5a28f6765a service neutron] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:41:08.904756 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32811c1-40b0-48e7-a80d-9b5a28f6765a service neutron] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:41:08.931780 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:41:09.011729 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74abc51-fe66-4684-b3bf-82df1d4a0e13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:41:09.012711 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74abc51-fe66-4684-b3bf-82df1d4a0e13 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:41:09.012996 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74abc51-fe66-4684-b3bf-82df1d4a0e13 admin admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:41:09.013276 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74abc51-fe66-4684-b3bf-82df1d4a0e13 admin admin] PATH_INFO: `/v3/users/165833ceb4c84f76bd128b726d0e31d9` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:41:09.014552 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b74abc51-fe66-4684-b3bf-82df1d4a0e13 admin admin] RBAC: Authorizing `identity:delete_user(user_id=165833ceb4c84f76bd128b726d0e31d9)` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:41:09.016520 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b74abc51-fe66-4684-b3bf-82df1d4a0e13 admin admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:41:09.064936 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-b74abc51-fe66-4684-b3bf-82df1d4a0e13 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '165833ceb4c84f76bd128b726d0e31d9'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:41:09.076582 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-b74abc51-fe66-4684-b3bf-82df1d4a0e13 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '165833ceb4c84f76bd128b726d0e31d9'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:41:09.084684 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-b74abc51-fe66-4684-b3bf-82df1d4a0e13 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '165833ceb4c84f76bd128b726d0e31d9'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:41:09.085354 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-b74abc51-fe66-4684-b3bf-82df1d4a0e13 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '165833ceb4c84f76bd128b726d0e31d9'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:41:09.085949 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-b74abc51-fe66-4684-b3bf-82df1d4a0e13 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '165833ceb4c84f76bd128b726d0e31d9'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:41:09.099789 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 984/1951] 10.222.0.57 () {58 vars in 1223 bytes} [Fri Jul 1 16:41:08 2022] DELETE /identity/v3/users/165833ceb4c84f76bd128b726d0e31d9 => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 16:41:09.570367 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4cae08-1b3b-4536-ae4f-2dfafdd61a55 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:41:09.572728 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-ae4cae08-1b3b-4536-ae4f-2dfafdd61a55 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:41:09.605061 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4cae08-1b3b-4536-ae4f-2dfafdd61a55 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:41:09.681018 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c49af72-1ebb-430e-b9d2-9748dc4e102d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:41:09.682503 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c49af72-1ebb-430e-b9d2-9748dc4e102d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:41:09.684006 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c49af72-1ebb-430e-b9d2-9748dc4e102d admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:41:09.684006 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c49af72-1ebb-430e-b9d2-9748dc4e102d admin admin] PATH_INFO: `/v3/projects/023fbbb1d8354bf39a57448345b1c59b` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:41:09.685509 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c49af72-1ebb-430e-b9d2-9748dc4e102d admin admin] RBAC: Authorizing `identity:delete_project(project_id=023fbbb1d8354bf39a57448345b1c59b)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:41:09.687653 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c49af72-1ebb-430e-b9d2-9748dc4e102d admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:41:09.754520 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-2c49af72-1ebb-430e-b9d2-9748dc4e102d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '023fbbb1d8354bf39a57448345b1c59b', 'request_id': 'req-2c49af72-1ebb-430e-b9d2-9748dc4e102d'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:41:09.760072 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-2c49af72-1ebb-430e-b9d2-9748dc4e102d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '023fbbb1d8354bf39a57448345b1c59b', 'request_id': 'req-2c49af72-1ebb-430e-b9d2-9748dc4e102d'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:41:09.766202 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-2c49af72-1ebb-430e-b9d2-9748dc4e102d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '023fbbb1d8354bf39a57448345b1c59b', 'request_id': 'req-2c49af72-1ebb-430e-b9d2-9748dc4e102d'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:41:09.779694 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-2c49af72-1ebb-430e-b9d2-9748dc4e102d admin admin] The token cache is being invalidate because project 023fbbb1d8354bf39a57448345b1c59b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80311) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 01 16:41:09.779989 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-2c49af72-1ebb-430e-b9d2-9748dc4e102d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:41:09.780594 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-2c49af72-1ebb-430e-b9d2-9748dc4e102d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:41:09.782398 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 968/1952] 10.222.0.57 () {58 vars in 1232 bytes} [Fri Jul 1 16:41:09 2022] DELETE /identity/v3/projects/023fbbb1d8354bf39a57448345b1c59b => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 16:41:09.788410 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647a10aa-9d4b-454d-82d9-20454df258b3 None None] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:41:09.788746 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647a10aa-9d4b-454d-82d9-20454df258b3 None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:41:09.789341 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647a10aa-9d4b-454d-82d9-20454df258b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:41:09.808375 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.auth.core [None req-647a10aa-9d4b-454d-82d9-20454df258b3 None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80312) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:41:09.809502 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-647a10aa-9d4b-454d-82d9-20454df258b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:41:09.872854 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 985/1953] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 1 16:41:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 16:41:09.881298 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f03cb12-6188-4b7a-8b99-793136bcdf44 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:41:09.881653 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f03cb12-6188-4b7a-8b99-793136bcdf44 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:41:09.881935 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f03cb12-6188-4b7a-8b99-793136bcdf44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:41:09.900958 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-3f03cb12-6188-4b7a-8b99-793136bcdf44 None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:41:09.902152 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-3f03cb12-6188-4b7a-8b99-793136bcdf44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:41:09.910222 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 969/1954] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 1 16:41:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 16:41:09.914459 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647a10aa-9d4b-454d-82d9-20454df258b3 None None] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:41:09.925218 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647a10aa-9d4b-454d-82d9-20454df258b3 None None] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:41:09.929920 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45eb452-8d40-4d4c-94b3-38d7da800b4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:41:09.930627 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45eb452-8d40-4d4c-94b3-38d7da800b4f admin admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:41:09.930919 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45eb452-8d40-4d4c-94b3-38d7da800b4f admin admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:41:09.931204 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45eb452-8d40-4d4c-94b3-38d7da800b4f admin admin] PATH_INFO: `/v3/domains` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:41:09.932072 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c45eb452-8d40-4d4c-94b3-38d7da800b4f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80312) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:41:09.932400 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c45eb452-8d40-4d4c-94b3-38d7da800b4f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:41:09.934007 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c45eb452-8d40-4d4c-94b3-38d7da800b4f admin admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:41:09.946458 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 986/1955] 10.222.0.57 () {58 vars in 1152 bytes} [Fri Jul 1 16:41:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:41:09.950822 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f03cb12-6188-4b7a-8b99-793136bcdf44 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:41:09.964612 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f03cb12-6188-4b7a-8b99-793136bcdf44 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:41:09.969892 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596215cf-8c5e-4b58-aadc-b4e14b877ec5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:41:09.971245 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596215cf-8c5e-4b58-aadc-b4e14b877ec5 admin admin] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:41:09.971245 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596215cf-8c5e-4b58-aadc-b4e14b877ec5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:41:09.971473 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596215cf-8c5e-4b58-aadc-b4e14b877ec5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:41:09.972259 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-596215cf-8c5e-4b58-aadc-b4e14b877ec5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:41:09.974266 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-596215cf-8c5e-4b58-aadc-b4e14b877ec5 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:41:10.006435 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 970/1956] 10.222.0.57 () {60 vars in 1152 bytes} [Fri Jul 1 16:41:09 2022] POST /identity/v3/projects => generated 377 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 01 16:41:10.009342 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45eb452-8d40-4d4c-94b3-38d7da800b4f admin admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:41:10.020739 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45eb452-8d40-4d4c-94b3-38d7da800b4f admin admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:41:10.041262 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:41:10.084743 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a581370-a5fa-4e3d-a82c-16edb52d9580 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:41:10.085435 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a581370-a5fa-4e3d-a82c-16edb52d9580 admin admin] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:41:10.085876 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a581370-a5fa-4e3d-a82c-16edb52d9580 admin admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:41:10.086002 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a581370-a5fa-4e3d-a82c-16edb52d9580 admin admin] PATH_INFO: `/v3/users` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:41:10.086738 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a581370-a5fa-4e3d-a82c-16edb52d9580 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:41:10.088643 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a581370-a5fa-4e3d-a82c-16edb52d9580 admin admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:41:10.109677 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 987/1957] 10.222.0.57 () {60 vars in 1143 bytes} [Fri Jul 1 16:41:10 2022] POST /identity/v3/users => generated 289 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 01 16:41:10.118095 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596215cf-8c5e-4b58-aadc-b4e14b877ec5 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:41:10.128968 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596215cf-8c5e-4b58-aadc-b4e14b877ec5 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:41:10.133545 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02bb5192-6e70-42ad-8a3a-1e4b5753c8d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:41:10.134197 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bb5192-6e70-42ad-8a3a-1e4b5753c8d7 admin admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:41:10.134515 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bb5192-6e70-42ad-8a3a-1e4b5753c8d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:41:10.134793 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bb5192-6e70-42ad-8a3a-1e4b5753c8d7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:41:10.135524 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02bb5192-6e70-42ad-8a3a-1e4b5753c8d7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:41:10.137104 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02bb5192-6e70-42ad-8a3a-1e4b5753c8d7 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:41:10.149751 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 971/1958] 10.222.0.57 () {58 vars in 1121 bytes} [Fri Jul 1 16:41:10 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 16:41:10.153840 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a581370-a5fa-4e3d-a82c-16edb52d9580 admin admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:41:10.165049 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a581370-a5fa-4e3d-a82c-16edb52d9580 admin admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:41:10.169644 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf15df3-8944-4ec3-a1be-dcded5c05986 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:41:10.170302 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf15df3-8944-4ec3-a1be-dcded5c05986 admin admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:41:10.170622 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf15df3-8944-4ec3-a1be-dcded5c05986 admin admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:41:10.170904 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf15df3-8944-4ec3-a1be-dcded5c05986 admin admin] PATH_INFO: `/v3/roles` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:41:10.171633 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf15df3-8944-4ec3-a1be-dcded5c05986 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:41:10.173253 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf15df3-8944-4ec3-a1be-dcded5c05986 admin admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:41:10.183337 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 988/1959] 10.222.0.57 () {58 vars in 1121 bytes} [Fri Jul 1 16:41:10 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 16:41:10.187408 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02bb5192-6e70-42ad-8a3a-1e4b5753c8d7 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:41:10.198174 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02bb5192-6e70-42ad-8a3a-1e4b5753c8d7 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:41:10.202849 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9e74f1-0ed2-4ec1-a06a-2159f573b435 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:41:10.203733 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9e74f1-0ed2-4ec1-a06a-2159f573b435 admin admin] REQUEST_METHOD: `PUT` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:41:10.204048 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9e74f1-0ed2-4ec1-a06a-2159f573b435 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:41:10.204331 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9e74f1-0ed2-4ec1-a06a-2159f573b435 admin admin] PATH_INFO: `/v3/projects/9ad8767852aa4ca09da7c5af2e412e91/users/a16472cb7e2c4cf58cbcdc372549b219/roles/983be40bdab04c478eaa90c6b6a46b14` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:41:10.216418 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:DELETE=12,keystone:SELECT=85,keystone:INSERT=3 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:41:10.221527 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9e74f1-0ed2-4ec1-a06a-2159f573b435 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9ad8767852aa4ca09da7c5af2e412e91, user_id=a16472cb7e2c4cf58cbcdc372549b219, role_id=983be40bdab04c478eaa90c6b6a46b14)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:41:10.224419 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9e74f1-0ed2-4ec1-a06a-2159f573b435 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:41:10.238802 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 972/1960] 10.222.0.57 () {60 vars in 1482 bytes} [Fri Jul 1 16:41:10 2022] PUT /identity/v3/projects/9ad8767852aa4ca09da7c5af2e412e91/users/a16472cb7e2c4cf58cbcdc372549b219/roles/983be40bdab04c478eaa90c6b6a46b14 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 01 16:41:10.249378 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79ce3ee-ba36-4ebf-bea5-9cd1394383b6 None None] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:41:10.249740 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79ce3ee-ba36-4ebf-bea5-9cd1394383b6 None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:41:10.250019 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79ce3ee-ba36-4ebf-bea5-9cd1394383b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:41:10.282727 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.auth.core [None req-d79ce3ee-ba36-4ebf-bea5-9cd1394383b6 None None] MFA Rules not processed for user `a16472cb7e2c4cf58cbcdc372549b219`. Rule list: `[]` (Enabled: `True`). {{(pid=80312) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:41:10.283954 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-d79ce3ee-ba36-4ebf-bea5-9cd1394383b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:41:10.348303 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 989/1961] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 1 16:41:10 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 16:41:10.352879 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9e74f1-0ed2-4ec1-a06a-2159f573b435 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:41:10.413423 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9e74f1-0ed2-4ec1-a06a-2159f573b435 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:41:10.417972 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0dcef8-f991-46d1-b822-07d4bb7abbfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:41:10.418652 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0dcef8-f991-46d1-b822-07d4bb7abbfb admin admin] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:41:10.418936 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0dcef8-f991-46d1-b822-07d4bb7abbfb admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:41:10.419211 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0dcef8-f991-46d1-b822-07d4bb7abbfb admin admin] PATH_INFO: `/v3/projects` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:41:10.419893 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae0dcef8-f991-46d1-b822-07d4bb7abbfb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:41:10.421777 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae0dcef8-f991-46d1-b822-07d4bb7abbfb admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:41:10.447648 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 973/1962] 10.222.0.57 () {60 vars in 1152 bytes} [Fri Jul 1 16:41:10 2022] POST /identity/v3/projects => generated 377 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 16:41:10.451863 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79ce3ee-ba36-4ebf-bea5-9cd1394383b6 None None] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:41:10.463140 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79ce3ee-ba36-4ebf-bea5-9cd1394383b6 None None] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:41:10.529435 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261d7569-18c3-4c33-918e-a12bb78d0c5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:41:10.530118 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261d7569-18c3-4c33-918e-a12bb78d0c5d admin admin] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:41:10.530440 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261d7569-18c3-4c33-918e-a12bb78d0c5d admin admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:41:10.530721 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261d7569-18c3-4c33-918e-a12bb78d0c5d admin admin] PATH_INFO: `/v3/users` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:41:10.531437 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-261d7569-18c3-4c33-918e-a12bb78d0c5d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:41:10.533790 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-261d7569-18c3-4c33-918e-a12bb78d0c5d admin admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:41:10.557431 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 990/1963] 10.222.0.57 () {60 vars in 1143 bytes} [Fri Jul 1 16:41:10 2022] POST /identity/v3/users => generated 289 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 16:41:10.561154 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0dcef8-f991-46d1-b822-07d4bb7abbfb admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:41:10.574209 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0dcef8-f991-46d1-b822-07d4bb7abbfb admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:41:10.579044 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b159f0db-b11d-4b88-ad9d-b43581c2f944 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:41:10.579743 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b159f0db-b11d-4b88-ad9d-b43581c2f944 admin admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:41:10.580065 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b159f0db-b11d-4b88-ad9d-b43581c2f944 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:41:10.580891 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b159f0db-b11d-4b88-ad9d-b43581c2f944 admin admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:41:10.581871 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b159f0db-b11d-4b88-ad9d-b43581c2f944 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:41:10.583500 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b159f0db-b11d-4b88-ad9d-b43581c2f944 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:41:10.598945 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 974/1964] 10.222.0.57 () {58 vars in 1121 bytes} [Fri Jul 1 16:41:10 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 16:41:10.600050 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261d7569-18c3-4c33-918e-a12bb78d0c5d admin admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:41:10.609979 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261d7569-18c3-4c33-918e-a12bb78d0c5d admin admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:41:10.616113 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0515f101-619a-4681-9fd7-72d6c10acc5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:41:10.616798 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0515f101-619a-4681-9fd7-72d6c10acc5d admin admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:41:10.617083 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0515f101-619a-4681-9fd7-72d6c10acc5d admin admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:41:10.617515 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0515f101-619a-4681-9fd7-72d6c10acc5d admin admin] PATH_INFO: `/v3/roles` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:41:10.618258 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0515f101-619a-4681-9fd7-72d6c10acc5d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:41:10.619910 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0515f101-619a-4681-9fd7-72d6c10acc5d admin admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:41:10.633980 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 991/1965] 10.222.0.57 () {58 vars in 1121 bytes} [Fri Jul 1 16:41:10 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 01 16:41:10.635475 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b159f0db-b11d-4b88-ad9d-b43581c2f944 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:41:10.647148 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b159f0db-b11d-4b88-ad9d-b43581c2f944 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:41:10.652093 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01be955-d99e-440f-9552-a5accec10345 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:41:10.652985 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01be955-d99e-440f-9552-a5accec10345 admin admin] REQUEST_METHOD: `PUT` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:41:10.653388 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01be955-d99e-440f-9552-a5accec10345 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:41:10.653673 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01be955-d99e-440f-9552-a5accec10345 admin admin] PATH_INFO: `/v3/projects/c80547bc713f43eba2a8314e2fb05c9e/users/4287b46443664cc29d4160af62bffb69/roles/983be40bdab04c478eaa90c6b6a46b14` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:41:10.671776 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e01be955-d99e-440f-9552-a5accec10345 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c80547bc713f43eba2a8314e2fb05c9e, user_id=4287b46443664cc29d4160af62bffb69, role_id=983be40bdab04c478eaa90c6b6a46b14)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:41:10.675813 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e01be955-d99e-440f-9552-a5accec10345 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:41:10.692197 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 975/1966] 10.222.0.57 () {60 vars in 1482 bytes} [Fri Jul 1 16:41:10 2022] PUT /identity/v3/projects/c80547bc713f43eba2a8314e2fb05c9e/users/4287b46443664cc29d4160af62bffb69/roles/983be40bdab04c478eaa90c6b6a46b14 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 16:41:10.703911 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9690cd42-94e3-44d5-a8f1-94be4d5456b3 None None] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:41:10.704298 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9690cd42-94e3-44d5-a8f1-94be4d5456b3 None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:41:10.704658 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9690cd42-94e3-44d5-a8f1-94be4d5456b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:41:10.726526 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:41:10.735606 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.auth.core [None req-9690cd42-94e3-44d5-a8f1-94be4d5456b3 None None] MFA Rules not processed for user `4287b46443664cc29d4160af62bffb69`. Rule list: `[]` (Enabled: `True`). {{(pid=80312) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:41:10.737919 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-9690cd42-94e3-44d5-a8f1-94be4d5456b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:41:10.815817 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 992/1967] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 1 16:41:10 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 16:41:10.820584 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01be955-d99e-440f-9552-a5accec10345 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:41:10.917364 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01be955-d99e-440f-9552-a5accec10345 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:41:10.922986 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6f1b5a-c751-4325-9628-586ded6c3da0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:41:10.924281 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6f1b5a-c751-4325-9628-586ded6c3da0 admin admin] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:41:10.925177 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6f1b5a-c751-4325-9628-586ded6c3da0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:41:10.925462 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6f1b5a-c751-4325-9628-586ded6c3da0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:41:10.926178 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb6f1b5a-c751-4325-9628-586ded6c3da0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:41:10.928123 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb6f1b5a-c751-4325-9628-586ded6c3da0 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:41:10.956374 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 976/1968] 10.222.0.57 () {60 vars in 1152 bytes} [Fri Jul 1 16:41:10 2022] POST /identity/v3/projects => generated 379 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 16:41:10.960843 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9690cd42-94e3-44d5-a8f1-94be4d5456b3 None None] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:41:10.972562 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9690cd42-94e3-44d5-a8f1-94be4d5456b3 None None] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:41:11.029564 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bde8fe1-3f80-4f9e-a941-5bc0e8d20364 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:41:11.030224 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bde8fe1-3f80-4f9e-a941-5bc0e8d20364 admin admin] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:41:11.030541 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bde8fe1-3f80-4f9e-a941-5bc0e8d20364 admin admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:41:11.030820 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bde8fe1-3f80-4f9e-a941-5bc0e8d20364 admin admin] PATH_INFO: `/v3/users` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:41:11.031536 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bde8fe1-3f80-4f9e-a941-5bc0e8d20364 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:41:11.034329 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bde8fe1-3f80-4f9e-a941-5bc0e8d20364 admin admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:41:11.060019 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 993/1969] 10.222.0.57 () {60 vars in 1143 bytes} [Fri Jul 1 16:41:10 2022] POST /identity/v3/users => generated 296 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 16:41:11.060764 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6f1b5a-c751-4325-9628-586ded6c3da0 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:41:11.072383 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6f1b5a-c751-4325-9628-586ded6c3da0 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:41:11.077450 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943c6d5d-1fe7-47d7-8f9d-fce3c265c357 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:41:11.078185 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943c6d5d-1fe7-47d7-8f9d-fce3c265c357 admin admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:41:11.078584 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943c6d5d-1fe7-47d7-8f9d-fce3c265c357 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:41:11.079031 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943c6d5d-1fe7-47d7-8f9d-fce3c265c357 admin admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:41:11.079846 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943c6d5d-1fe7-47d7-8f9d-fce3c265c357 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:41:11.081543 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943c6d5d-1fe7-47d7-8f9d-fce3c265c357 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:41:11.093112 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 977/1970] 10.222.0.57 () {58 vars in 1121 bytes} [Fri Jul 1 16:41:11 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 16:41:11.097780 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bde8fe1-3f80-4f9e-a941-5bc0e8d20364 admin admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:41:11.108617 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bde8fe1-3f80-4f9e-a941-5bc0e8d20364 admin admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:41:11.114268 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146c4e40-6c45-45e2-a9a8-f4374c5903bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:41:11.115718 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146c4e40-6c45-45e2-a9a8-f4374c5903bf admin admin] REQUEST_METHOD: `PUT` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:41:11.115718 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146c4e40-6c45-45e2-a9a8-f4374c5903bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:41:11.115912 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146c4e40-6c45-45e2-a9a8-f4374c5903bf admin admin] PATH_INFO: `/v3/projects/4cfc31dc318d4d6fa2eb10ede6b749fe/users/d8562a3075b14cc48237c2b6fe4c4a7a/roles/84ee739146934c1ab32c501ff8093644` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:41:11.131957 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-146c4e40-6c45-45e2-a9a8-f4374c5903bf admin admin] RBAC: Authorizing `identity:create_grant(project_id=4cfc31dc318d4d6fa2eb10ede6b749fe, user_id=d8562a3075b14cc48237c2b6fe4c4a7a, role_id=84ee739146934c1ab32c501ff8093644)` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:41:11.134768 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-146c4e40-6c45-45e2-a9a8-f4374c5903bf admin admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:41:11.151166 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 994/1971] 10.222.0.57 () {60 vars in 1482 bytes} [Fri Jul 1 16:41:11 2022] PUT /identity/v3/projects/4cfc31dc318d4d6fa2eb10ede6b749fe/users/d8562a3075b14cc48237c2b6fe4c4a7a/roles/84ee739146934c1ab32c501ff8093644 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 01 16:41:11.155058 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943c6d5d-1fe7-47d7-8f9d-fce3c265c357 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:41:11.166704 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943c6d5d-1fe7-47d7-8f9d-fce3c265c357 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:41:11.183079 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:41:11.235960 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde73452-f1ca-4cec-8c9f-c9dba79b8799 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:41:11.236858 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde73452-f1ca-4cec-8c9f-c9dba79b8799 admin admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:41:11.237337 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde73452-f1ca-4cec-8c9f-c9dba79b8799 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:41:11.237683 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde73452-f1ca-4cec-8c9f-c9dba79b8799 admin admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:41:11.238497 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde73452-f1ca-4cec-8c9f-c9dba79b8799 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:41:11.240128 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde73452-f1ca-4cec-8c9f-c9dba79b8799 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:41:11.254024 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 978/1972] 10.222.0.57 () {58 vars in 1121 bytes} [Fri Jul 1 16:41:11 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 16:41:11.255763 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146c4e40-6c45-45e2-a9a8-f4374c5903bf admin admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:41:11.267076 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146c4e40-6c45-45e2-a9a8-f4374c5903bf admin admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:41:11.273295 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53953664-17cd-4c15-9758-a7881ab526f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:41:11.274411 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53953664-17cd-4c15-9758-a7881ab526f8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:41:11.274698 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53953664-17cd-4c15-9758-a7881ab526f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:41:11.274974 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53953664-17cd-4c15-9758-a7881ab526f8 admin admin] PATH_INFO: `/v3/projects/4cfc31dc318d4d6fa2eb10ede6b749fe/users/d8562a3075b14cc48237c2b6fe4c4a7a/roles/84ee739146934c1ab32c501ff8093644` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:41:11.276978 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53953664-17cd-4c15-9758-a7881ab526f8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4cfc31dc318d4d6fa2eb10ede6b749fe, user_id=d8562a3075b14cc48237c2b6fe4c4a7a, role_id=84ee739146934c1ab32c501ff8093644)` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:41:11.279658 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53953664-17cd-4c15-9758-a7881ab526f8 admin admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:41:11.291519 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 995/1973] 10.222.0.57 () {60 vars in 1482 bytes} [Fri Jul 1 16:41:11 2022] PUT /identity/v3/projects/4cfc31dc318d4d6fa2eb10ede6b749fe/users/d8562a3075b14cc48237c2b6fe4c4a7a/roles/84ee739146934c1ab32c501ff8093644 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 16:41:11.301638 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9212d620-cb69-4562-b3de-cae7de09915a None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:41:11.301937 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9212d620-cb69-4562-b3de-cae7de09915a None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:41:11.302216 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9212d620-cb69-4562-b3de-cae7de09915a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:41:11.328989 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-9212d620-cb69-4562-b3de-cae7de09915a None None] MFA Rules not processed for user `d8562a3075b14cc48237c2b6fe4c4a7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:41:11.330816 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-9212d620-cb69-4562-b3de-cae7de09915a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:41:11.400868 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 979/1974] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 1 16:41:11 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 16:41:11.413642 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53953664-17cd-4c15-9758-a7881ab526f8 admin admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:41:11.415337 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-53953664-17cd-4c15-9758-a7881ab526f8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:41:11.475791 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53953664-17cd-4c15-9758-a7881ab526f8 admin admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:41:11.479466 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9f65c1-14cf-4c8d-a48e-c64086851fa6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbddf971a2354bbdbb9da1d63f3abf88', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0788e6725a84e57a4174284c1845755', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:41:11.480118 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9f65c1-14cf-4c8d-a48e-c64086851fa6 service cinder] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:41:11.480401 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9f65c1-14cf-4c8d-a48e-c64086851fa6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:41:11.480676 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9f65c1-14cf-4c8d-a48e-c64086851fa6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:41:11.488602 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:41:11.491958 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e9f65c1-14cf-4c8d-a48e-c64086851fa6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:41:11.493593 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e9f65c1-14cf-4c8d-a48e-c64086851fa6 service cinder] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:41:11.512589 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 996/1975] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 1 16:41:11 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 16:41:11.583151 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9212d620-cb69-4562-b3de-cae7de09915a None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:41:11.593883 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9212d620-cb69-4562-b3de-cae7de09915a None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:41:11.597932 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584f5ef2-7fb1-4642-955a-4e3c7cd8fe0f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbddf971a2354bbdbb9da1d63f3abf88', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0788e6725a84e57a4174284c1845755', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:41:11.598533 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584f5ef2-7fb1-4642-955a-4e3c7cd8fe0f service cinder] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:41:11.598823 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584f5ef2-7fb1-4642-955a-4e3c7cd8fe0f service cinder] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:41:11.599100 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584f5ef2-7fb1-4642-955a-4e3c7cd8fe0f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:41:11.664425 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-584f5ef2-7fb1-4642-955a-4e3c7cd8fe0f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:41:11.665854 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-584f5ef2-7fb1-4642-955a-4e3c7cd8fe0f service cinder] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:41:11.674970 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 980/1976] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 1 16:41:11 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 16:41:17.208653 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9f65c1-14cf-4c8d-a48e-c64086851fa6 service cinder] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:41:17.331275 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9f65c1-14cf-4c8d-a48e-c64086851fa6 service cinder] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:41:17.338248 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e297e7-c74b-4bd4-9d5b-83a8c45693a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:41:17.339476 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e297e7-c74b-4bd4-9d5b-83a8c45693a3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:41:17.339769 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e297e7-c74b-4bd4-9d5b-83a8c45693a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:41:17.340051 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e297e7-c74b-4bd4-9d5b-83a8c45693a3 admin admin] PATH_INFO: `/v3/users/a16472cb7e2c4cf58cbcdc372549b219` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:41:17.341649 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e297e7-c74b-4bd4-9d5b-83a8c45693a3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a16472cb7e2c4cf58cbcdc372549b219)` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:41:17.344612 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e297e7-c74b-4bd4-9d5b-83a8c45693a3 admin admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:41:17.411573 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-c2e297e7-c74b-4bd4-9d5b-83a8c45693a3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a16472cb7e2c4cf58cbcdc372549b219'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:41:17.427511 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-c2e297e7-c74b-4bd4-9d5b-83a8c45693a3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a16472cb7e2c4cf58cbcdc372549b219'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:41:17.436551 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-c2e297e7-c74b-4bd4-9d5b-83a8c45693a3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a16472cb7e2c4cf58cbcdc372549b219'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:41:17.437409 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-c2e297e7-c74b-4bd4-9d5b-83a8c45693a3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a16472cb7e2c4cf58cbcdc372549b219'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:41:17.438087 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-c2e297e7-c74b-4bd4-9d5b-83a8c45693a3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a16472cb7e2c4cf58cbcdc372549b219'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:41:17.455787 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 997/1977] 10.222.0.57 () {58 vars in 1223 bytes} [Fri Jul 1 16:41:17 2022] DELETE /identity/v3/users/a16472cb7e2c4cf58cbcdc372549b219 => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 16:41:17.460937 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73315ae9-4cb4-4bc7-8364-f90df32d564e None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:41:17.461702 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73315ae9-4cb4-4bc7-8364-f90df32d564e None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:41:17.462054 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73315ae9-4cb4-4bc7-8364-f90df32d564e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:41:17.487136 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-73315ae9-4cb4-4bc7-8364-f90df32d564e None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:41:17.490281 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-73315ae9-4cb4-4bc7-8364-f90df32d564e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:41:17.534114 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:41:17.575137 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 981/1978] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 1 16:41:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 16:41:17.587056 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e297e7-c74b-4bd4-9d5b-83a8c45693a3 admin admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:41:17.588572 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-c2e297e7-c74b-4bd4-9d5b-83a8c45693a3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:41:17.655839 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e297e7-c74b-4bd4-9d5b-83a8c45693a3 admin admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:41:17.660330 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38a983f-d765-411b-a73a-5c7873a57ecf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ed200ce29684c5f80354fa9a8638fe2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0788e6725a84e57a4174284c1845755', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:41:17.660926 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38a983f-d765-411b-a73a-5c7873a57ecf service neutron] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:41:17.661212 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38a983f-d765-411b-a73a-5c7873a57ecf service neutron] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:41:17.661496 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38a983f-d765-411b-a73a-5c7873a57ecf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:41:17.671299 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b38a983f-d765-411b-a73a-5c7873a57ecf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:41:17.672713 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b38a983f-d765-411b-a73a-5c7873a57ecf service neutron] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:41:17.686404 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 998/1979] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 1 16:41:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 16:41:18.243425 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73315ae9-4cb4-4bc7-8364-f90df32d564e None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:41:18.245377 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-73315ae9-4cb4-4bc7-8364-f90df32d564e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:41:18.292998 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73315ae9-4cb4-4bc7-8364-f90df32d564e None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:41:18.307079 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efca37ef-48cf-454f-bb9f-7076f3e5e051 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:41:18.308916 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efca37ef-48cf-454f-bb9f-7076f3e5e051 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:41:18.309486 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efca37ef-48cf-454f-bb9f-7076f3e5e051 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:41:18.309877 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efca37ef-48cf-454f-bb9f-7076f3e5e051 admin admin] PATH_INFO: `/v3/projects/9ad8767852aa4ca09da7c5af2e412e91` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:41:18.311656 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efca37ef-48cf-454f-bb9f-7076f3e5e051 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9ad8767852aa4ca09da7c5af2e412e91)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:41:18.314478 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efca37ef-48cf-454f-bb9f-7076f3e5e051 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:41:18.411065 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-efca37ef-48cf-454f-bb9f-7076f3e5e051 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9ad8767852aa4ca09da7c5af2e412e91', 'request_id': 'req-efca37ef-48cf-454f-bb9f-7076f3e5e051'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:41:18.417718 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-efca37ef-48cf-454f-bb9f-7076f3e5e051 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9ad8767852aa4ca09da7c5af2e412e91', 'request_id': 'req-efca37ef-48cf-454f-bb9f-7076f3e5e051'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:41:18.429203 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-efca37ef-48cf-454f-bb9f-7076f3e5e051 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9ad8767852aa4ca09da7c5af2e412e91', 'request_id': 'req-efca37ef-48cf-454f-bb9f-7076f3e5e051'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:41:18.447152 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-efca37ef-48cf-454f-bb9f-7076f3e5e051 admin admin] The token cache is being invalidate because project 9ad8767852aa4ca09da7c5af2e412e91 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80311) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 01 16:41:18.447596 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-efca37ef-48cf-454f-bb9f-7076f3e5e051 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:41:18.448587 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-efca37ef-48cf-454f-bb9f-7076f3e5e051 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:41:18.450872 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 982/1980] 10.222.0.57 () {58 vars in 1232 bytes} [Fri Jul 1 16:41:18 2022] DELETE /identity/v3/projects/9ad8767852aa4ca09da7c5af2e412e91 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 16:41:18.454607 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38a983f-d765-411b-a73a-5c7873a57ecf service neutron] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:41:18.465803 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38a983f-d765-411b-a73a-5c7873a57ecf service neutron] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:41:18.482914 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:41:18.643113 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173f0723-17ea-4e90-a9e5-43bc022e8635 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:41:18.644609 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173f0723-17ea-4e90-a9e5-43bc022e8635 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:41:18.648494 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173f0723-17ea-4e90-a9e5-43bc022e8635 admin admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:41:18.649233 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173f0723-17ea-4e90-a9e5-43bc022e8635 admin admin] PATH_INFO: `/v3/users/4287b46443664cc29d4160af62bffb69` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:41:18.653743 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-173f0723-17ea-4e90-a9e5-43bc022e8635 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4287b46443664cc29d4160af62bffb69)` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:41:18.659518 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-173f0723-17ea-4e90-a9e5-43bc022e8635 admin admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:41:18.693725 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efca37ef-48cf-454f-bb9f-7076f3e5e051 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:41:18.697493 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-efca37ef-48cf-454f-bb9f-7076f3e5e051 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:41:18.741033 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efca37ef-48cf-454f-bb9f-7076f3e5e051 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:41:18.753865 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2407bdf5-1b08-4b84-864d-f478747d243a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:41:18.755237 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2407bdf5-1b08-4b84-864d-f478747d243a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:41:18.756623 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2407bdf5-1b08-4b84-864d-f478747d243a admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:41:18.757061 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2407bdf5-1b08-4b84-864d-f478747d243a admin admin] PATH_INFO: `/v3/users/3b430c67c85e437e810d2d967a3af5c0` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:41:18.763359 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2407bdf5-1b08-4b84-864d-f478747d243a admin admin] RBAC: Authorizing `identity:delete_user(user_id=3b430c67c85e437e810d2d967a3af5c0)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:41:18.766660 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2407bdf5-1b08-4b84-864d-f478747d243a admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:41:18.774840 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-173f0723-17ea-4e90-a9e5-43bc022e8635 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4287b46443664cc29d4160af62bffb69'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:41:18.831016 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-173f0723-17ea-4e90-a9e5-43bc022e8635 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4287b46443664cc29d4160af62bffb69'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:41:18.856397 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-173f0723-17ea-4e90-a9e5-43bc022e8635 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4287b46443664cc29d4160af62bffb69'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:41:18.859668 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-173f0723-17ea-4e90-a9e5-43bc022e8635 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4287b46443664cc29d4160af62bffb69'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:41:18.860820 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-173f0723-17ea-4e90-a9e5-43bc022e8635 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4287b46443664cc29d4160af62bffb69'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:41:18.888919 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 999/1981] 10.222.0.57 () {58 vars in 1223 bytes} [Fri Jul 1 16:41:18 2022] DELETE /identity/v3/users/4287b46443664cc29d4160af62bffb69 => generated 0 bytes in 435 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 16:41:18.910870 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-2407bdf5-1b08-4b84-864d-f478747d243a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3b430c67c85e437e810d2d967a3af5c0'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:41:18.914117 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-2407bdf5-1b08-4b84-864d-f478747d243a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3b430c67c85e437e810d2d967a3af5c0'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:41:18.918061 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-2407bdf5-1b08-4b84-864d-f478747d243a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3b430c67c85e437e810d2d967a3af5c0'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:41:18.943481 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-2407bdf5-1b08-4b84-864d-f478747d243a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3b430c67c85e437e810d2d967a3af5c0'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:41:18.973162 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-2407bdf5-1b08-4b84-864d-f478747d243a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3b430c67c85e437e810d2d967a3af5c0'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:41:18.999574 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 983/1982] 10.222.0.57 () {58 vars in 1223 bytes} [Fri Jul 1 16:41:18 2022] DELETE /identity/v3/users/3b430c67c85e437e810d2d967a3af5c0 => generated 0 bytes in 308 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 01 16:41:19.005374 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86901b20-cc76-4ada-9dfa-8abdcebbfb8e None None] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:41:19.005668 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86901b20-cc76-4ada-9dfa-8abdcebbfb8e None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:41:19.006003 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86901b20-cc76-4ada-9dfa-8abdcebbfb8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:41:19.053312 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.auth.core [None req-86901b20-cc76-4ada-9dfa-8abdcebbfb8e None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80312) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:41:19.054578 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-86901b20-cc76-4ada-9dfa-8abdcebbfb8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:41:19.233247 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1000/1983] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 1 16:41:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 228 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 16:41:19.251394 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2407bdf5-1b08-4b84-864d-f478747d243a admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:41:19.253525 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-2407bdf5-1b08-4b84-864d-f478747d243a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:41:19.318698 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:41:19.382871 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2407bdf5-1b08-4b84-864d-f478747d243a admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:41:19.393209 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708509d4-aaa2-4917-be69-eccb72f3c80d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ed200ce29684c5f80354fa9a8638fe2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0788e6725a84e57a4174284c1845755', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:41:19.393892 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708509d4-aaa2-4917-be69-eccb72f3c80d service neutron] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:41:19.394179 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708509d4-aaa2-4917-be69-eccb72f3c80d service neutron] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:41:19.395950 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708509d4-aaa2-4917-be69-eccb72f3c80d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:41:19.418511 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-708509d4-aaa2-4917-be69-eccb72f3c80d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:41:19.420439 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-708509d4-aaa2-4917-be69-eccb72f3c80d service neutron] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:41:19.441536 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86901b20-cc76-4ada-9dfa-8abdcebbfb8e None None] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:41:19.443038 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-86901b20-cc76-4ada-9dfa-8abdcebbfb8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:41:19.445358 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 984/1984] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 1 16:41:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 194 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 16:41:19.467207 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86901b20-cc76-4ada-9dfa-8abdcebbfb8e None None] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:41:19.479830 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ac434a-8005-4753-b386-ec832c3d4e7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:41:19.480862 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ac434a-8005-4753-b386-ec832c3d4e7f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:41:19.481155 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ac434a-8005-4753-b386-ec832c3d4e7f admin admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:41:19.482000 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ac434a-8005-4753-b386-ec832c3d4e7f admin admin] PATH_INFO: `/v3/projects/c80547bc713f43eba2a8314e2fb05c9e` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:41:19.485595 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ac434a-8005-4753-b386-ec832c3d4e7f admin admin] RBAC: Authorizing `identity:delete_project(project_id=c80547bc713f43eba2a8314e2fb05c9e)` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:41:19.491580 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ac434a-8005-4753-b386-ec832c3d4e7f admin admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:41:19.610143 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:41:19.641151 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-40ac434a-8005-4753-b386-ec832c3d4e7f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c80547bc713f43eba2a8314e2fb05c9e', 'request_id': 'req-40ac434a-8005-4753-b386-ec832c3d4e7f'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:41:19.655361 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-40ac434a-8005-4753-b386-ec832c3d4e7f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c80547bc713f43eba2a8314e2fb05c9e', 'request_id': 'req-40ac434a-8005-4753-b386-ec832c3d4e7f'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:41:19.666657 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-40ac434a-8005-4753-b386-ec832c3d4e7f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c80547bc713f43eba2a8314e2fb05c9e', 'request_id': 'req-40ac434a-8005-4753-b386-ec832c3d4e7f'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:41:19.700702 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-40ac434a-8005-4753-b386-ec832c3d4e7f admin admin] The token cache is being invalidate because project c80547bc713f43eba2a8314e2fb05c9e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80312) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 01 16:41:19.700919 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-40ac434a-8005-4753-b386-ec832c3d4e7f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:41:19.703003 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-40ac434a-8005-4753-b386-ec832c3d4e7f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:41:19.708475 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1001/1985] 10.222.0.57 () {58 vars in 1232 bytes} [Fri Jul 1 16:41:19 2022] DELETE /identity/v3/projects/c80547bc713f43eba2a8314e2fb05c9e => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 01 16:41:19.712836 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708509d4-aaa2-4917-be69-eccb72f3c80d service neutron] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:41:19.725324 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708509d4-aaa2-4917-be69-eccb72f3c80d service neutron] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:41:19.853178 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1f7b5b-422c-4fc1-95be-75ef7dee8deb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:41:19.854242 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1f7b5b-422c-4fc1-95be-75ef7dee8deb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:41:19.854821 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1f7b5b-422c-4fc1-95be-75ef7dee8deb admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:41:19.855109 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1f7b5b-422c-4fc1-95be-75ef7dee8deb admin admin] PATH_INFO: `/v3/users/d8562a3075b14cc48237c2b6fe4c4a7a` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:41:19.858268 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c1f7b5b-422c-4fc1-95be-75ef7dee8deb admin admin] RBAC: Authorizing `identity:delete_user(user_id=d8562a3075b14cc48237c2b6fe4c4a7a)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:41:19.860294 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c1f7b5b-422c-4fc1-95be-75ef7dee8deb admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:41:20.020303 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-3c1f7b5b-422c-4fc1-95be-75ef7dee8deb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd8562a3075b14cc48237c2b6fe4c4a7a'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:41:20.021267 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-3c1f7b5b-422c-4fc1-95be-75ef7dee8deb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd8562a3075b14cc48237c2b6fe4c4a7a'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:41:20.022055 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-3c1f7b5b-422c-4fc1-95be-75ef7dee8deb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd8562a3075b14cc48237c2b6fe4c4a7a'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:41:20.035580 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-3c1f7b5b-422c-4fc1-95be-75ef7dee8deb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd8562a3075b14cc48237c2b6fe4c4a7a'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:41:20.067380 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-3c1f7b5b-422c-4fc1-95be-75ef7dee8deb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd8562a3075b14cc48237c2b6fe4c4a7a'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:41:20.102224 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 985/1986] 10.222.0.57 () {58 vars in 1223 bytes} [Fri Jul 1 16:41:19 2022] DELETE /identity/v3/users/d8562a3075b14cc48237c2b6fe4c4a7a => generated 0 bytes in 390 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 16:41:20.187024 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ac434a-8005-4753-b386-ec832c3d4e7f admin admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:41:20.197211 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-40ac434a-8005-4753-b386-ec832c3d4e7f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:41:20.218194 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ac434a-8005-4753-b386-ec832c3d4e7f admin admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:41:20.294102 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4bc5f67-ef5a-4639-8ac8-19127b8f236a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:41:20.295015 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4bc5f67-ef5a-4639-8ac8-19127b8f236a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:41:20.295302 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4bc5f67-ef5a-4639-8ac8-19127b8f236a admin admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:41:20.295581 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4bc5f67-ef5a-4639-8ac8-19127b8f236a admin admin] PATH_INFO: `/v3/projects/c30ee4688f3c4f40b62c810dab8c7d82` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:41:20.296952 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4bc5f67-ef5a-4639-8ac8-19127b8f236a admin admin] RBAC: Authorizing `identity:delete_project(project_id=c30ee4688f3c4f40b62c810dab8c7d82)` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:41:20.298852 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4bc5f67-ef5a-4639-8ac8-19127b8f236a admin admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:41:20.369286 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-c4bc5f67-ef5a-4639-8ac8-19127b8f236a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c30ee4688f3c4f40b62c810dab8c7d82', 'request_id': 'req-c4bc5f67-ef5a-4639-8ac8-19127b8f236a'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:41:20.384534 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-c4bc5f67-ef5a-4639-8ac8-19127b8f236a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c30ee4688f3c4f40b62c810dab8c7d82', 'request_id': 'req-c4bc5f67-ef5a-4639-8ac8-19127b8f236a'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:41:20.395751 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-c4bc5f67-ef5a-4639-8ac8-19127b8f236a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c30ee4688f3c4f40b62c810dab8c7d82', 'request_id': 'req-c4bc5f67-ef5a-4639-8ac8-19127b8f236a'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:41:20.427944 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-c4bc5f67-ef5a-4639-8ac8-19127b8f236a admin admin] The token cache is being invalidate because project c30ee4688f3c4f40b62c810dab8c7d82 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80312) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 01 16:41:20.427944 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-c4bc5f67-ef5a-4639-8ac8-19127b8f236a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:41:20.431248 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-c4bc5f67-ef5a-4639-8ac8-19127b8f236a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:41:20.433070 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1002/1987] 10.222.0.57 () {58 vars in 1232 bytes} [Fri Jul 1 16:41:20 2022] DELETE /identity/v3/projects/c30ee4688f3c4f40b62c810dab8c7d82 => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 16:41:20.443210 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1f7b5b-422c-4fc1-95be-75ef7dee8deb admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:41:20.445237 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-3c1f7b5b-422c-4fc1-95be-75ef7dee8deb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:41:20.461076 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1f7b5b-422c-4fc1-95be-75ef7dee8deb admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:41:20.555998 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:41:20.577471 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b944cc-bcc6-4439-8a67-6e53d36dd667 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:41:20.579138 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b944cc-bcc6-4439-8a67-6e53d36dd667 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:41:20.579702 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b944cc-bcc6-4439-8a67-6e53d36dd667 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:41:20.581597 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b944cc-bcc6-4439-8a67-6e53d36dd667 admin admin] PATH_INFO: `/v3/users/e83a9e97966d493ba56184049aa097d5` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:41:20.582561 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b944cc-bcc6-4439-8a67-6e53d36dd667 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e83a9e97966d493ba56184049aa097d5)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:41:20.585928 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b944cc-bcc6-4439-8a67-6e53d36dd667 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:41:20.657565 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-48b944cc-bcc6-4439-8a67-6e53d36dd667 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e83a9e97966d493ba56184049aa097d5'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:41:20.659118 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-48b944cc-bcc6-4439-8a67-6e53d36dd667 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e83a9e97966d493ba56184049aa097d5'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:41:20.659839 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-48b944cc-bcc6-4439-8a67-6e53d36dd667 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e83a9e97966d493ba56184049aa097d5'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:41:20.669960 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-48b944cc-bcc6-4439-8a67-6e53d36dd667 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e83a9e97966d493ba56184049aa097d5'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:41:20.693310 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-48b944cc-bcc6-4439-8a67-6e53d36dd667 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e83a9e97966d493ba56184049aa097d5'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:41:20.730548 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 986/1988] 10.222.0.57 () {58 vars in 1223 bytes} [Fri Jul 1 16:41:20 2022] DELETE /identity/v3/users/e83a9e97966d493ba56184049aa097d5 => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 16:41:21.071662 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4bc5f67-ef5a-4639-8ac8-19127b8f236a admin admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:41:21.073763 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-c4bc5f67-ef5a-4639-8ac8-19127b8f236a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:41:21.091269 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4bc5f67-ef5a-4639-8ac8-19127b8f236a admin admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:41:21.154933 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:41:21.167970 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76cfa4b-3444-4487-aca2-212991a1e6a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:41:21.169014 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76cfa4b-3444-4487-aca2-212991a1e6a4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:41:21.169529 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76cfa4b-3444-4487-aca2-212991a1e6a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:41:21.169635 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76cfa4b-3444-4487-aca2-212991a1e6a4 admin admin] PATH_INFO: `/v3/projects/1395a12992e9423286ddc2ad5265c260` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:41:21.171069 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76cfa4b-3444-4487-aca2-212991a1e6a4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1395a12992e9423286ddc2ad5265c260)` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:41:21.174213 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76cfa4b-3444-4487-aca2-212991a1e6a4 admin admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:41:21.234291 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-b76cfa4b-3444-4487-aca2-212991a1e6a4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1395a12992e9423286ddc2ad5265c260', 'request_id': 'req-b76cfa4b-3444-4487-aca2-212991a1e6a4'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:41:21.245024 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-b76cfa4b-3444-4487-aca2-212991a1e6a4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1395a12992e9423286ddc2ad5265c260', 'request_id': 'req-b76cfa4b-3444-4487-aca2-212991a1e6a4'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:41:21.251863 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-b76cfa4b-3444-4487-aca2-212991a1e6a4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1395a12992e9423286ddc2ad5265c260', 'request_id': 'req-b76cfa4b-3444-4487-aca2-212991a1e6a4'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:41:21.266231 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-b76cfa4b-3444-4487-aca2-212991a1e6a4 admin admin] The token cache is being invalidate because project 1395a12992e9423286ddc2ad5265c260 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80312) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 01 16:41:21.266803 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-b76cfa4b-3444-4487-aca2-212991a1e6a4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:41:21.268056 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-b76cfa4b-3444-4487-aca2-212991a1e6a4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:41:21.270538 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1003/1989] 10.222.0.57 () {58 vars in 1232 bytes} [Fri Jul 1 16:41:21 2022] DELETE /identity/v3/projects/1395a12992e9423286ddc2ad5265c260 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 16:41:21.275749 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b944cc-bcc6-4439-8a67-6e53d36dd667 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:41:21.277910 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-48b944cc-bcc6-4439-8a67-6e53d36dd667 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:41:21.293601 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b944cc-bcc6-4439-8a67-6e53d36dd667 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:41:21.357894 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76cfa4b-3444-4487-aca2-212991a1e6a4 admin admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:41:21.359442 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-b76cfa4b-3444-4487-aca2-212991a1e6a4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:41:21.378047 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e026dc5-8c35-41e4-9ce9-30dda022571e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:41:21.379106 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e026dc5-8c35-41e4-9ce9-30dda022571e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:41:21.379462 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e026dc5-8c35-41e4-9ce9-30dda022571e admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:41:21.379814 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e026dc5-8c35-41e4-9ce9-30dda022571e admin admin] PATH_INFO: `/v3/users/2f78e619ff21465ea0d4d6a19e640b6a` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:41:21.381166 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e026dc5-8c35-41e4-9ce9-30dda022571e admin admin] RBAC: Authorizing `identity:delete_user(user_id=2f78e619ff21465ea0d4d6a19e640b6a)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:41:21.383231 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e026dc5-8c35-41e4-9ce9-30dda022571e admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:41:21.449850 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-2e026dc5-8c35-41e4-9ce9-30dda022571e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2f78e619ff21465ea0d4d6a19e640b6a'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:41:21.451151 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-2e026dc5-8c35-41e4-9ce9-30dda022571e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2f78e619ff21465ea0d4d6a19e640b6a'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:41:21.452131 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-2e026dc5-8c35-41e4-9ce9-30dda022571e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2f78e619ff21465ea0d4d6a19e640b6a'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:41:21.452753 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76cfa4b-3444-4487-aca2-212991a1e6a4 admin admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:41:21.463334 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f2a8c3-b68a-4ffa-bc70-9bb9696e9ce7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:41:21.463651 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-2e026dc5-8c35-41e4-9ce9-30dda022571e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2f78e619ff21465ea0d4d6a19e640b6a'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:41:21.464756 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f2a8c3-b68a-4ffa-bc70-9bb9696e9ce7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:41:21.465784 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f2a8c3-b68a-4ffa-bc70-9bb9696e9ce7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:41:21.466320 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f2a8c3-b68a-4ffa-bc70-9bb9696e9ce7 admin admin] PATH_INFO: `/v3/projects/4cfc31dc318d4d6fa2eb10ede6b749fe` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:41:21.468663 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f2a8c3-b68a-4ffa-bc70-9bb9696e9ce7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4cfc31dc318d4d6fa2eb10ede6b749fe)` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:41:21.472529 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f2a8c3-b68a-4ffa-bc70-9bb9696e9ce7 admin admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:41:21.481086 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-2e026dc5-8c35-41e4-9ce9-30dda022571e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2f78e619ff21465ea0d4d6a19e640b6a'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:41:21.482670 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:41:21.499874 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 987/1990] 10.222.0.57 () {58 vars in 1223 bytes} [Fri Jul 1 16:41:21 2022] DELETE /identity/v3/users/2f78e619ff21465ea0d4d6a19e640b6a => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 16:41:21.535304 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-a1f2a8c3-b68a-4ffa-bc70-9bb9696e9ce7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4cfc31dc318d4d6fa2eb10ede6b749fe', 'request_id': 'req-a1f2a8c3-b68a-4ffa-bc70-9bb9696e9ce7'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:41:21.544170 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-a1f2a8c3-b68a-4ffa-bc70-9bb9696e9ce7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4cfc31dc318d4d6fa2eb10ede6b749fe', 'request_id': 'req-a1f2a8c3-b68a-4ffa-bc70-9bb9696e9ce7'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:41:21.551109 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-a1f2a8c3-b68a-4ffa-bc70-9bb9696e9ce7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4cfc31dc318d4d6fa2eb10ede6b749fe', 'request_id': 'req-a1f2a8c3-b68a-4ffa-bc70-9bb9696e9ce7'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:41:21.564734 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-a1f2a8c3-b68a-4ffa-bc70-9bb9696e9ce7 admin admin] The token cache is being invalidate because project 4cfc31dc318d4d6fa2eb10ede6b749fe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80312) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 01 16:41:21.565031 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-a1f2a8c3-b68a-4ffa-bc70-9bb9696e9ce7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:41:21.565906 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-a1f2a8c3-b68a-4ffa-bc70-9bb9696e9ce7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:41:21.567901 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1004/1991] 10.222.0.57 () {58 vars in 1232 bytes} [Fri Jul 1 16:41:21 2022] DELETE /identity/v3/projects/4cfc31dc318d4d6fa2eb10ede6b749fe => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 01 16:41:21.936580 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e026dc5-8c35-41e4-9ce9-30dda022571e admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:41:21.938168 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-2e026dc5-8c35-41e4-9ce9-30dda022571e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:41:22.035371 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e026dc5-8c35-41e4-9ce9-30dda022571e admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:41:22.094570 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f32042d-235b-42f7-8986-65e0a6594619 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:41:22.095569 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f32042d-235b-42f7-8986-65e0a6594619 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:41:22.096024 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f32042d-235b-42f7-8986-65e0a6594619 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:41:22.096314 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f32042d-235b-42f7-8986-65e0a6594619 admin admin] PATH_INFO: `/v3/projects/10bf42ede27e4c9c8a2faa4dd348fbff` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:41:22.097566 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f32042d-235b-42f7-8986-65e0a6594619 admin admin] RBAC: Authorizing `identity:delete_project(project_id=10bf42ede27e4c9c8a2faa4dd348fbff)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:41:22.099786 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f32042d-235b-42f7-8986-65e0a6594619 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:41:22.149249 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-3f32042d-235b-42f7-8986-65e0a6594619 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '10bf42ede27e4c9c8a2faa4dd348fbff', 'request_id': 'req-3f32042d-235b-42f7-8986-65e0a6594619'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:41:22.154825 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-3f32042d-235b-42f7-8986-65e0a6594619 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '10bf42ede27e4c9c8a2faa4dd348fbff', 'request_id': 'req-3f32042d-235b-42f7-8986-65e0a6594619'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:41:22.162100 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-3f32042d-235b-42f7-8986-65e0a6594619 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '10bf42ede27e4c9c8a2faa4dd348fbff', 'request_id': 'req-3f32042d-235b-42f7-8986-65e0a6594619'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:41:22.175449 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-3f32042d-235b-42f7-8986-65e0a6594619 admin admin] The token cache is being invalidate because project 10bf42ede27e4c9c8a2faa4dd348fbff was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80311) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 01 16:41:22.175788 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-3f32042d-235b-42f7-8986-65e0a6594619 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:41:22.176427 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-3f32042d-235b-42f7-8986-65e0a6594619 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:41:22.178007 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 988/1992] 10.222.0.57 () {58 vars in 1232 bytes} [Fri Jul 1 16:41:21 2022] DELETE /identity/v3/projects/10bf42ede27e4c9c8a2faa4dd348fbff => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 16:41:22.183391 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b25646-1c64-45ed-8b01-e19c2b71cc9a None None] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:41:22.183865 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b25646-1c64-45ed-8b01-e19c2b71cc9a None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:41:22.184085 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b25646-1c64-45ed-8b01-e19c2b71cc9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:41:22.202990 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.auth.core [None req-42b25646-1c64-45ed-8b01-e19c2b71cc9a None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80312) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:41:22.204193 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-42b25646-1c64-45ed-8b01-e19c2b71cc9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:41:22.210636 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:41:22.278104 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1005/1993] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 1 16:41:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 16:41:22.286078 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8b0814-01a3-4346-ae99-1e24a561b2a0 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:41:22.286394 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8b0814-01a3-4346-ae99-1e24a561b2a0 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:41:22.286683 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8b0814-01a3-4346-ae99-1e24a561b2a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:41:22.305148 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-6e8b0814-01a3-4346-ae99-1e24a561b2a0 None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:41:22.306382 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-6e8b0814-01a3-4346-ae99-1e24a561b2a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:41:22.315195 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 989/1994] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 1 16:41:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 16:41:22.319470 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b25646-1c64-45ed-8b01-e19c2b71cc9a None None] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:41:22.329010 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b25646-1c64-45ed-8b01-e19c2b71cc9a None None] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:41:22.335609 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a30a2f-f4a0-4f23-9015-2dba43800618 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:41:22.335609 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a30a2f-f4a0-4f23-9015-2dba43800618 admin admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:41:22.335852 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a30a2f-f4a0-4f23-9015-2dba43800618 admin admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:41:22.336103 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a30a2f-f4a0-4f23-9015-2dba43800618 admin admin] PATH_INFO: `/v3/domains` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:41:22.336841 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a30a2f-f4a0-4f23-9015-2dba43800618 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80312) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:41:22.337152 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a30a2f-f4a0-4f23-9015-2dba43800618 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:41:22.338792 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a30a2f-f4a0-4f23-9015-2dba43800618 admin admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:41:22.353437 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1006/1995] 10.222.0.57 () {58 vars in 1152 bytes} [Fri Jul 1 16:41:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:41:22.357711 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8b0814-01a3-4346-ae99-1e24a561b2a0 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:41:22.368897 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8b0814-01a3-4346-ae99-1e24a561b2a0 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:41:22.373801 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d925714f-22c0-49f1-bc3c-34c29014b7ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:41:22.374596 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d925714f-22c0-49f1-bc3c-34c29014b7ab admin admin] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:41:22.374890 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d925714f-22c0-49f1-bc3c-34c29014b7ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:41:22.375174 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d925714f-22c0-49f1-bc3c-34c29014b7ab admin admin] PATH_INFO: `/v3/projects` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:41:22.375889 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d925714f-22c0-49f1-bc3c-34c29014b7ab admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:41:22.378809 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d925714f-22c0-49f1-bc3c-34c29014b7ab admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:41:22.398720 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:41:22.407961 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 990/1996] 10.222.0.57 () {60 vars in 1152 bytes} [Fri Jul 1 16:41:22 2022] POST /identity/v3/projects => generated 373 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 16:41:22.412157 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a30a2f-f4a0-4f23-9015-2dba43800618 admin admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:41:22.422893 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a30a2f-f4a0-4f23-9015-2dba43800618 admin admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:41:22.481188 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651969d7-68ce-4d59-8530-22c4c60bae4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:41:22.481871 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651969d7-68ce-4d59-8530-22c4c60bae4c admin admin] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:41:22.482155 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651969d7-68ce-4d59-8530-22c4c60bae4c admin admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:41:22.482477 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651969d7-68ce-4d59-8530-22c4c60bae4c admin admin] PATH_INFO: `/v3/users` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:41:22.483203 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-651969d7-68ce-4d59-8530-22c4c60bae4c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:41:22.485148 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-651969d7-68ce-4d59-8530-22c4c60bae4c admin admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:41:22.507312 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1007/1997] 10.222.0.57 () {60 vars in 1143 bytes} [Fri Jul 1 16:41:22 2022] POST /identity/v3/users => generated 287 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 01 16:41:22.511367 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d925714f-22c0-49f1-bc3c-34c29014b7ab admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:41:22.522443 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d925714f-22c0-49f1-bc3c-34c29014b7ab admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:41:22.527374 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ba25f9-7d0d-4e40-95ac-f2ccd9dd7afe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:41:22.528045 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ba25f9-7d0d-4e40-95ac-f2ccd9dd7afe admin admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:41:22.528333 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ba25f9-7d0d-4e40-95ac-f2ccd9dd7afe admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:41:22.528614 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ba25f9-7d0d-4e40-95ac-f2ccd9dd7afe admin admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:41:22.529349 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ba25f9-7d0d-4e40-95ac-f2ccd9dd7afe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:41:22.530990 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ba25f9-7d0d-4e40-95ac-f2ccd9dd7afe admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:41:22.542272 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 991/1998] 10.222.0.57 () {58 vars in 1121 bytes} [Fri Jul 1 16:41:22 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 16:41:22.546146 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651969d7-68ce-4d59-8530-22c4c60bae4c admin admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:41:22.560035 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651969d7-68ce-4d59-8530-22c4c60bae4c admin admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:41:22.564875 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6233f1f6-304f-4158-ae31-3b71004c77b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:41:22.565546 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6233f1f6-304f-4158-ae31-3b71004c77b7 admin admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:41:22.565839 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6233f1f6-304f-4158-ae31-3b71004c77b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:41:22.566178 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6233f1f6-304f-4158-ae31-3b71004c77b7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:41:22.566886 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6233f1f6-304f-4158-ae31-3b71004c77b7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:41:22.568501 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6233f1f6-304f-4158-ae31-3b71004c77b7 admin admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:41:22.579368 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1008/1999] 10.222.0.57 () {58 vars in 1121 bytes} [Fri Jul 1 16:41:22 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 16:41:22.584133 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ba25f9-7d0d-4e40-95ac-f2ccd9dd7afe admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:41:22.596407 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ba25f9-7d0d-4e40-95ac-f2ccd9dd7afe admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:41:22.601200 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41516678-ba8a-4633-9200-7ed485168200 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:41:22.602034 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41516678-ba8a-4633-9200-7ed485168200 admin admin] REQUEST_METHOD: `PUT` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:41:22.602310 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41516678-ba8a-4633-9200-7ed485168200 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:41:22.602631 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41516678-ba8a-4633-9200-7ed485168200 admin admin] PATH_INFO: `/v3/projects/1427b7f42f374e1a9782153fd2a9cd7b/users/d7fa77f8e9fd4c71a6b1faa84a32c159/roles/983be40bdab04c478eaa90c6b6a46b14` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:41:22.619137 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41516678-ba8a-4633-9200-7ed485168200 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1427b7f42f374e1a9782153fd2a9cd7b, user_id=d7fa77f8e9fd4c71a6b1faa84a32c159, role_id=983be40bdab04c478eaa90c6b6a46b14)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:41:22.621922 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41516678-ba8a-4633-9200-7ed485168200 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:41:22.635941 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 992/2000] 10.222.0.57 () {60 vars in 1482 bytes} [Fri Jul 1 16:41:22 2022] PUT /identity/v3/projects/1427b7f42f374e1a9782153fd2a9cd7b/users/d7fa77f8e9fd4c71a6b1faa84a32c159/roles/983be40bdab04c478eaa90c6b6a46b14 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 16:41:22.647635 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4a0933-b12a-47c9-b885-638788d2b8f2 None None] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:41:22.647635 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4a0933-b12a-47c9-b885-638788d2b8f2 None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:41:22.647878 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4a0933-b12a-47c9-b885-638788d2b8f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:41:22.679403 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.auth.core [None req-0b4a0933-b12a-47c9-b885-638788d2b8f2 None None] MFA Rules not processed for user `d7fa77f8e9fd4c71a6b1faa84a32c159`. Rule list: `[]` (Enabled: `True`). {{(pid=80312) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:41:22.681132 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-0b4a0933-b12a-47c9-b885-638788d2b8f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:41:22.760012 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1009/2001] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 1 16:41:22 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 16:41:22.772043 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41516678-ba8a-4633-9200-7ed485168200 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:41:22.774072 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-41516678-ba8a-4633-9200-7ed485168200 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:41:22.860624 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41516678-ba8a-4633-9200-7ed485168200 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:41:22.865576 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fcaec81-62a8-45c4-a554-d17f1b6222cd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbddf971a2354bbdbb9da1d63f3abf88', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0788e6725a84e57a4174284c1845755', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:41:22.866448 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcaec81-62a8-45c4-a554-d17f1b6222cd service cinder] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:41:22.866884 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcaec81-62a8-45c4-a554-d17f1b6222cd service cinder] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:41:22.867182 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcaec81-62a8-45c4-a554-d17f1b6222cd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:41:22.890364 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fcaec81-62a8-45c4-a554-d17f1b6222cd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:41:22.891872 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fcaec81-62a8-45c4-a554-d17f1b6222cd service cinder] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:41:22.902774 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 993/2002] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 1 16:41:22 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 16:41:22.931250 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4a0933-b12a-47c9-b885-638788d2b8f2 None None] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:41:22.941005 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:41:22.999083 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4a0933-b12a-47c9-b885-638788d2b8f2 None None] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:41:23.004298 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df090bf8-9c30-4c62-9f9f-8125c2bdbd75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:41:23.005212 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df090bf8-9c30-4c62-9f9f-8125c2bdbd75 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:41:23.005417 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df090bf8-9c30-4c62-9f9f-8125c2bdbd75 admin admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:41:23.005691 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df090bf8-9c30-4c62-9f9f-8125c2bdbd75 admin admin] PATH_INFO: `/v3/users/d7fa77f8e9fd4c71a6b1faa84a32c159` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:41:23.006947 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df090bf8-9c30-4c62-9f9f-8125c2bdbd75 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d7fa77f8e9fd4c71a6b1faa84a32c159)` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:41:23.008885 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df090bf8-9c30-4c62-9f9f-8125c2bdbd75 admin admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:41:23.057822 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-df090bf8-9c30-4c62-9f9f-8125c2bdbd75 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd7fa77f8e9fd4c71a6b1faa84a32c159'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:41:23.077140 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-df090bf8-9c30-4c62-9f9f-8125c2bdbd75 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd7fa77f8e9fd4c71a6b1faa84a32c159'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:41:23.085460 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-df090bf8-9c30-4c62-9f9f-8125c2bdbd75 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd7fa77f8e9fd4c71a6b1faa84a32c159'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:41:23.086182 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-df090bf8-9c30-4c62-9f9f-8125c2bdbd75 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd7fa77f8e9fd4c71a6b1faa84a32c159'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:41:23.086822 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-df090bf8-9c30-4c62-9f9f-8125c2bdbd75 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd7fa77f8e9fd4c71a6b1faa84a32c159'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:41:23.100327 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1010/2003] 10.222.0.57 () {58 vars in 1223 bytes} [Fri Jul 1 16:41:22 2022] DELETE /identity/v3/users/d7fa77f8e9fd4c71a6b1faa84a32c159 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 16:41:23.104990 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3cc0ea6-9b61-46bc-a13e-579c1bce547c None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:41:23.105516 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3cc0ea6-9b61-46bc-a13e-579c1bce547c None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:41:23.105627 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3cc0ea6-9b61-46bc-a13e-579c1bce547c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:41:23.123973 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-d3cc0ea6-9b61-46bc-a13e-579c1bce547c None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:41:23.125182 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-d3cc0ea6-9b61-46bc-a13e-579c1bce547c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:41:23.193868 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 994/2004] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 1 16:41:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 16:41:23.203514 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df090bf8-9c30-4c62-9f9f-8125c2bdbd75 admin admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:41:23.205480 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-df090bf8-9c30-4c62-9f9f-8125c2bdbd75 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:41:23.264728 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df090bf8-9c30-4c62-9f9f-8125c2bdbd75 admin admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:41:23.269780 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a73a09-df28-4c5c-8dee-8c7ebf113f40 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ed200ce29684c5f80354fa9a8638fe2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0788e6725a84e57a4174284c1845755', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:41:23.270475 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a73a09-df28-4c5c-8dee-8c7ebf113f40 service neutron] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:41:23.270764 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a73a09-df28-4c5c-8dee-8c7ebf113f40 service neutron] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:41:23.271043 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a73a09-df28-4c5c-8dee-8c7ebf113f40 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:41:23.280632 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a73a09-df28-4c5c-8dee-8c7ebf113f40 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:41:23.282072 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a73a09-df28-4c5c-8dee-8c7ebf113f40 service neutron] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:41:23.292249 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1011/2005] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 1 16:41:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 16:41:23.569657 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3cc0ea6-9b61-46bc-a13e-579c1bce547c None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:41:23.571179 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-d3cc0ea6-9b61-46bc-a13e-579c1bce547c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:41:23.585488 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3cc0ea6-9b61-46bc-a13e-579c1bce547c None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:41:23.590999 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde75099-2050-40f0-a35d-df705e3da1c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:41:23.592590 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde75099-2050-40f0-a35d-df705e3da1c2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:41:23.592903 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde75099-2050-40f0-a35d-df705e3da1c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:41:23.593217 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde75099-2050-40f0-a35d-df705e3da1c2 admin admin] PATH_INFO: `/v3/projects/1427b7f42f374e1a9782153fd2a9cd7b` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:41:23.595045 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde75099-2050-40f0-a35d-df705e3da1c2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1427b7f42f374e1a9782153fd2a9cd7b)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:41:23.596873 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde75099-2050-40f0-a35d-df705e3da1c2 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:41:23.644394 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-bde75099-2050-40f0-a35d-df705e3da1c2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1427b7f42f374e1a9782153fd2a9cd7b', 'request_id': 'req-bde75099-2050-40f0-a35d-df705e3da1c2'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:41:23.649707 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-bde75099-2050-40f0-a35d-df705e3da1c2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1427b7f42f374e1a9782153fd2a9cd7b', 'request_id': 'req-bde75099-2050-40f0-a35d-df705e3da1c2'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:41:23.651537 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=93,keystone:INSERT=1,keystone:DELETE=6 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:41:23.656201 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-bde75099-2050-40f0-a35d-df705e3da1c2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1427b7f42f374e1a9782153fd2a9cd7b', 'request_id': 'req-bde75099-2050-40f0-a35d-df705e3da1c2'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:41:23.671862 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-bde75099-2050-40f0-a35d-df705e3da1c2 admin admin] The token cache is being invalidate because project 1427b7f42f374e1a9782153fd2a9cd7b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80311) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 01 16:41:23.672291 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-bde75099-2050-40f0-a35d-df705e3da1c2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:41:23.672787 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-bde75099-2050-40f0-a35d-df705e3da1c2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:41:23.675558 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 995/2006] 10.222.0.57 () {58 vars in 1232 bytes} [Fri Jul 1 16:41:23 2022] DELETE /identity/v3/projects/1427b7f42f374e1a9782153fd2a9cd7b => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 16:41:23.680622 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de116b4-682f-4ffc-a05b-f1f97289999f None None] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:41:23.680917 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de116b4-682f-4ffc-a05b-f1f97289999f None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:41:23.681207 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de116b4-682f-4ffc-a05b-f1f97289999f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:41:23.700491 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.auth.core [None req-0de116b4-682f-4ffc-a05b-f1f97289999f None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80312) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:41:23.701728 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-0de116b4-682f-4ffc-a05b-f1f97289999f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:41:23.726186 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:41:23.763135 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1012/2007] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 1 16:41:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 16:41:23.770983 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db76a1c-1920-4f38-ba2b-33182ac3f374 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:41:23.771275 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db76a1c-1920-4f38-ba2b-33182ac3f374 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:41:23.771556 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db76a1c-1920-4f38-ba2b-33182ac3f374 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:41:23.790624 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-6db76a1c-1920-4f38-ba2b-33182ac3f374 None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:41:23.791823 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-6db76a1c-1920-4f38-ba2b-33182ac3f374 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:41:23.800311 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 996/2008] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 1 16:41:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 16:41:23.805029 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de116b4-682f-4ffc-a05b-f1f97289999f None None] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:41:23.815117 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de116b4-682f-4ffc-a05b-f1f97289999f None None] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:41:23.819546 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785a9f74-1b11-4598-a01d-695cd5132a75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:41:23.820193 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785a9f74-1b11-4598-a01d-695cd5132a75 admin admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:41:23.820477 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785a9f74-1b11-4598-a01d-695cd5132a75 admin admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:41:23.820749 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785a9f74-1b11-4598-a01d-695cd5132a75 admin admin] PATH_INFO: `/v3/domains` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:41:23.821470 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785a9f74-1b11-4598-a01d-695cd5132a75 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80312) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:41:23.821776 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785a9f74-1b11-4598-a01d-695cd5132a75 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:41:23.823403 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785a9f74-1b11-4598-a01d-695cd5132a75 admin admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:41:23.835371 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1013/2009] 10.222.0.57 () {58 vars in 1152 bytes} [Fri Jul 1 16:41:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:41:23.839717 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db76a1c-1920-4f38-ba2b-33182ac3f374 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:41:23.850000 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db76a1c-1920-4f38-ba2b-33182ac3f374 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:41:23.854783 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d103641-7aed-465a-ace6-fcd7f7807bf9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:41:23.855448 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d103641-7aed-465a-ace6-fcd7f7807bf9 admin admin] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:41:23.855766 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d103641-7aed-465a-ace6-fcd7f7807bf9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:41:23.856068 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d103641-7aed-465a-ace6-fcd7f7807bf9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:41:23.856784 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d103641-7aed-465a-ace6-fcd7f7807bf9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:41:23.858884 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d103641-7aed-465a-ace6-fcd7f7807bf9 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:41:23.884986 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 997/2010] 10.222.0.57 () {60 vars in 1152 bytes} [Fri Jul 1 16:41:23 2022] POST /identity/v3/projects => generated 383 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 16:41:23.888970 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785a9f74-1b11-4598-a01d-695cd5132a75 admin admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:41:23.900203 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785a9f74-1b11-4598-a01d-695cd5132a75 admin admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:41:23.957921 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb0e3bb-1b4d-4c36-a76f-e18c49c96f08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:41:23.958715 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb0e3bb-1b4d-4c36-a76f-e18c49c96f08 admin admin] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:41:23.959007 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb0e3bb-1b4d-4c36-a76f-e18c49c96f08 admin admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:41:23.959850 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb0e3bb-1b4d-4c36-a76f-e18c49c96f08 admin admin] PATH_INFO: `/v3/users` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:41:23.960261 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb0e3bb-1b4d-4c36-a76f-e18c49c96f08 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:41:23.962204 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb0e3bb-1b4d-4c36-a76f-e18c49c96f08 admin admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:41:23.987072 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1014/2011] 10.222.0.57 () {60 vars in 1143 bytes} [Fri Jul 1 16:41:23 2022] POST /identity/v3/users => generated 292 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 16:41:23.988260 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d103641-7aed-465a-ace6-fcd7f7807bf9 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:41:23.998627 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d103641-7aed-465a-ace6-fcd7f7807bf9 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:41:24.004333 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00454f95-ea7f-4e10-a5ee-4904ff418a8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:41:24.005008 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00454f95-ea7f-4e10-a5ee-4904ff418a8b admin admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:41:24.005294 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00454f95-ea7f-4e10-a5ee-4904ff418a8b admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:41:24.005573 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00454f95-ea7f-4e10-a5ee-4904ff418a8b admin admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:41:24.006308 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00454f95-ea7f-4e10-a5ee-4904ff418a8b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:41:24.008033 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00454f95-ea7f-4e10-a5ee-4904ff418a8b admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:41:24.019115 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 998/2012] 10.222.0.57 () {58 vars in 1121 bytes} [Fri Jul 1 16:41:23 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 16:41:24.023112 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb0e3bb-1b4d-4c36-a76f-e18c49c96f08 admin admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:41:24.033665 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb0e3bb-1b4d-4c36-a76f-e18c49c96f08 admin admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:41:24.038733 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3dc6e71-7f28-469e-805d-0cbc790408a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:41:24.039414 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3dc6e71-7f28-469e-805d-0cbc790408a5 admin admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:41:24.039702 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3dc6e71-7f28-469e-805d-0cbc790408a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:41:24.039983 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3dc6e71-7f28-469e-805d-0cbc790408a5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:41:24.040774 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3dc6e71-7f28-469e-805d-0cbc790408a5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:41:24.042416 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3dc6e71-7f28-469e-805d-0cbc790408a5 admin admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:41:24.053743 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1015/2013] 10.222.0.57 () {58 vars in 1121 bytes} [Fri Jul 1 16:41:24 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 16:41:24.057856 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00454f95-ea7f-4e10-a5ee-4904ff418a8b admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:41:24.069388 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00454f95-ea7f-4e10-a5ee-4904ff418a8b admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:41:24.074400 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed837cb-130f-41a8-b0fc-a8aceefcc677 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:41:24.075297 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed837cb-130f-41a8-b0fc-a8aceefcc677 admin admin] REQUEST_METHOD: `PUT` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:41:24.075595 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed837cb-130f-41a8-b0fc-a8aceefcc677 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:41:24.075890 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed837cb-130f-41a8-b0fc-a8aceefcc677 admin admin] PATH_INFO: `/v3/projects/ef0cc672766c47759a088a398d95147f/users/c1ab0b52325c4ab387db034eb7dfcadb/roles/983be40bdab04c478eaa90c6b6a46b14` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:41:24.092188 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed837cb-130f-41a8-b0fc-a8aceefcc677 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ef0cc672766c47759a088a398d95147f, user_id=c1ab0b52325c4ab387db034eb7dfcadb, role_id=983be40bdab04c478eaa90c6b6a46b14)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:41:24.094918 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed837cb-130f-41a8-b0fc-a8aceefcc677 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:41:24.109452 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 999/2014] 10.222.0.57 () {60 vars in 1482 bytes} [Fri Jul 1 16:41:24 2022] PUT /identity/v3/projects/ef0cc672766c47759a088a398d95147f/users/c1ab0b52325c4ab387db034eb7dfcadb/roles/983be40bdab04c478eaa90c6b6a46b14 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 01 16:41:24.115266 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6a82f3-958d-4843-81b1-853931e33885 None None] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:41:24.115917 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6a82f3-958d-4843-81b1-853931e33885 None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:41:24.115917 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6a82f3-958d-4843-81b1-853931e33885 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:41:24.133673 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.auth.core [None req-bf6a82f3-958d-4843-81b1-853931e33885 None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80312) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:41:24.134928 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-bf6a82f3-958d-4843-81b1-853931e33885 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:41:24.194383 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1016/2015] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 1 16:41:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 16:41:24.204081 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed837cb-130f-41a8-b0fc-a8aceefcc677 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:41:24.206435 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-5ed837cb-130f-41a8-b0fc-a8aceefcc677 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:41:24.263957 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed837cb-130f-41a8-b0fc-a8aceefcc677 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:41:24.268451 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741cb4ce-8052-4888-b29e-0ef3b74821d0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ed200ce29684c5f80354fa9a8638fe2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0788e6725a84e57a4174284c1845755', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:41:24.269101 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741cb4ce-8052-4888-b29e-0ef3b74821d0 service neutron] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:41:24.269388 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741cb4ce-8052-4888-b29e-0ef3b74821d0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:41:24.269669 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741cb4ce-8052-4888-b29e-0ef3b74821d0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:41:24.280797 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-741cb4ce-8052-4888-b29e-0ef3b74821d0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:41:24.282234 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-741cb4ce-8052-4888-b29e-0ef3b74821d0 service neutron] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:41:24.293240 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1000/2016] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 1 16:41:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 16:41:26.210311 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6a82f3-958d-4843-81b1-853931e33885 None None] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:41:26.211915 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-bf6a82f3-958d-4843-81b1-853931e33885 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:41:26.225226 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6a82f3-958d-4843-81b1-853931e33885 None None] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:41:26.230678 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ffc586-ede1-4cdc-a883-f380d91892fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:41:26.231569 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ffc586-ede1-4cdc-a883-f380d91892fd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:41:26.231855 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ffc586-ede1-4cdc-a883-f380d91892fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:41:26.232134 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ffc586-ede1-4cdc-a883-f380d91892fd admin admin] PATH_INFO: `/v3/users/9ebe4c339d454eeb861a7f5e1ca53d20` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:41:26.233319 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ffc586-ede1-4cdc-a883-f380d91892fd admin admin] RBAC: Authorizing `identity:delete_user(user_id=9ebe4c339d454eeb861a7f5e1ca53d20)` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:41:26.235321 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ffc586-ede1-4cdc-a883-f380d91892fd admin admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:41:26.272748 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:41:26.289312 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-71ffc586-ede1-4cdc-a883-f380d91892fd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9ebe4c339d454eeb861a7f5e1ca53d20'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:41:26.304188 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-71ffc586-ede1-4cdc-a883-f380d91892fd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9ebe4c339d454eeb861a7f5e1ca53d20'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:41:26.311654 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-71ffc586-ede1-4cdc-a883-f380d91892fd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9ebe4c339d454eeb861a7f5e1ca53d20'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:41:26.312473 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-71ffc586-ede1-4cdc-a883-f380d91892fd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9ebe4c339d454eeb861a7f5e1ca53d20'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:41:26.313182 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-71ffc586-ede1-4cdc-a883-f380d91892fd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9ebe4c339d454eeb861a7f5e1ca53d20'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:41:26.329293 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1017/2017] 10.222.0.57 () {58 vars in 1223 bytes} [Fri Jul 1 16:41:26 2022] DELETE /identity/v3/users/9ebe4c339d454eeb861a7f5e1ca53d20 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 16:41:26.334325 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ecbd59-c799-4099-8753-2ecda750e3ff None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:41:26.338189 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ecbd59-c799-4099-8753-2ecda750e3ff None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:41:26.338504 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ecbd59-c799-4099-8753-2ecda750e3ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:41:26.371663 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-43ecbd59-c799-4099-8753-2ecda750e3ff None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:41:26.373080 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-43ecbd59-c799-4099-8753-2ecda750e3ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:41:26.463986 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1001/2018] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 1 16:41:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 01 16:41:26.481498 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ffc586-ede1-4cdc-a883-f380d91892fd admin admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:41:26.483155 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-71ffc586-ede1-4cdc-a883-f380d91892fd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:41:26.592685 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ffc586-ede1-4cdc-a883-f380d91892fd admin admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:41:26.601979 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae295434-b2f7-45ca-9d5f-c678e3c86298 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ed200ce29684c5f80354fa9a8638fe2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0788e6725a84e57a4174284c1845755', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:41:26.603135 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae295434-b2f7-45ca-9d5f-c678e3c86298 service neutron] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:41:26.603676 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae295434-b2f7-45ca-9d5f-c678e3c86298 service neutron] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:41:26.604057 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae295434-b2f7-45ca-9d5f-c678e3c86298 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:41:26.635830 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae295434-b2f7-45ca-9d5f-c678e3c86298 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:41:26.638047 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae295434-b2f7-45ca-9d5f-c678e3c86298 service neutron] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:41:26.651539 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1018/2019] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 1 16:41:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 16:41:27.809547 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ecbd59-c799-4099-8753-2ecda750e3ff None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:41:27.811631 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-43ecbd59-c799-4099-8753-2ecda750e3ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:41:27.827621 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ecbd59-c799-4099-8753-2ecda750e3ff None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:41:27.835272 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad14248a-d5bb-4965-bbb4-278222c87808 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:41:27.836158 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad14248a-d5bb-4965-bbb4-278222c87808 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:41:27.836445 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad14248a-d5bb-4965-bbb4-278222c87808 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:41:27.836722 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad14248a-d5bb-4965-bbb4-278222c87808 admin admin] PATH_INFO: `/v3/projects/0776ab3ff39c4bbc8e4ba19eb6a978b9` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:41:27.839278 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad14248a-d5bb-4965-bbb4-278222c87808 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0776ab3ff39c4bbc8e4ba19eb6a978b9)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:41:27.841582 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad14248a-d5bb-4965-bbb4-278222c87808 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:41:27.880744 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=95,keystone:INSERT=3,keystone:DELETE=2 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:41:27.917141 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-ad14248a-d5bb-4965-bbb4-278222c87808 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0776ab3ff39c4bbc8e4ba19eb6a978b9', 'request_id': 'req-ad14248a-d5bb-4965-bbb4-278222c87808'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:41:27.925083 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-ad14248a-d5bb-4965-bbb4-278222c87808 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0776ab3ff39c4bbc8e4ba19eb6a978b9', 'request_id': 'req-ad14248a-d5bb-4965-bbb4-278222c87808'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:41:27.932810 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-ad14248a-d5bb-4965-bbb4-278222c87808 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0776ab3ff39c4bbc8e4ba19eb6a978b9', 'request_id': 'req-ad14248a-d5bb-4965-bbb4-278222c87808'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:41:27.947351 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-ad14248a-d5bb-4965-bbb4-278222c87808 admin admin] The token cache is being invalidate because project 0776ab3ff39c4bbc8e4ba19eb6a978b9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80311) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 01 16:41:27.947645 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-ad14248a-d5bb-4965-bbb4-278222c87808 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:41:27.948271 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-ad14248a-d5bb-4965-bbb4-278222c87808 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:41:27.950082 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1002/2020] 10.222.0.57 () {58 vars in 1232 bytes} [Fri Jul 1 16:41:27 2022] DELETE /identity/v3/projects/0776ab3ff39c4bbc8e4ba19eb6a978b9 => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 16:41:27.954024 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae295434-b2f7-45ca-9d5f-c678e3c86298 service neutron] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:41:27.964258 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae295434-b2f7-45ca-9d5f-c678e3c86298 service neutron] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:41:28.026771 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64be04a-4e78-4e5b-ba0f-9d70655b7925 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:41:28.027660 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64be04a-4e78-4e5b-ba0f-9d70655b7925 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:41:28.029167 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64be04a-4e78-4e5b-ba0f-9d70655b7925 admin admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:41:28.029167 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64be04a-4e78-4e5b-ba0f-9d70655b7925 admin admin] PATH_INFO: `/v3/users/38179860d6754ad4b1eb8d59cb2d16c6` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:41:28.029915 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b64be04a-4e78-4e5b-ba0f-9d70655b7925 admin admin] RBAC: Authorizing `identity:delete_user(user_id=38179860d6754ad4b1eb8d59cb2d16c6)` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:41:28.031888 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b64be04a-4e78-4e5b-ba0f-9d70655b7925 admin admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:41:28.092493 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-b64be04a-4e78-4e5b-ba0f-9d70655b7925 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '38179860d6754ad4b1eb8d59cb2d16c6'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:41:28.099818 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:41:28.110262 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-b64be04a-4e78-4e5b-ba0f-9d70655b7925 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '38179860d6754ad4b1eb8d59cb2d16c6'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:41:28.122385 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-b64be04a-4e78-4e5b-ba0f-9d70655b7925 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '38179860d6754ad4b1eb8d59cb2d16c6'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:41:28.123131 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-b64be04a-4e78-4e5b-ba0f-9d70655b7925 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '38179860d6754ad4b1eb8d59cb2d16c6'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:41:28.123741 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-b64be04a-4e78-4e5b-ba0f-9d70655b7925 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '38179860d6754ad4b1eb8d59cb2d16c6'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:41:28.141829 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1019/2021] 10.222.0.57 () {58 vars in 1223 bytes} [Fri Jul 1 16:41:27 2022] DELETE /identity/v3/users/38179860d6754ad4b1eb8d59cb2d16c6 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 16:41:28.590308 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad14248a-d5bb-4965-bbb4-278222c87808 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:41:28.592168 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-ad14248a-d5bb-4965-bbb4-278222c87808 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:41:28.622662 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad14248a-d5bb-4965-bbb4-278222c87808 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:41:28.705385 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab0ba34-baac-4483-8807-fecc726f7218 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:41:28.706547 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab0ba34-baac-4483-8807-fecc726f7218 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:41:28.706835 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab0ba34-baac-4483-8807-fecc726f7218 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:41:28.707165 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab0ba34-baac-4483-8807-fecc726f7218 admin admin] PATH_INFO: `/v3/projects/b36385de9bfd4ba691c28ab5550aa7dc` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:41:28.708572 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab0ba34-baac-4483-8807-fecc726f7218 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b36385de9bfd4ba691c28ab5550aa7dc)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:41:28.711638 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab0ba34-baac-4483-8807-fecc726f7218 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:41:28.793380 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-2ab0ba34-baac-4483-8807-fecc726f7218 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b36385de9bfd4ba691c28ab5550aa7dc', 'request_id': 'req-2ab0ba34-baac-4483-8807-fecc726f7218'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:41:28.803924 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-2ab0ba34-baac-4483-8807-fecc726f7218 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b36385de9bfd4ba691c28ab5550aa7dc', 'request_id': 'req-2ab0ba34-baac-4483-8807-fecc726f7218'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:41:28.810681 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-2ab0ba34-baac-4483-8807-fecc726f7218 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b36385de9bfd4ba691c28ab5550aa7dc', 'request_id': 'req-2ab0ba34-baac-4483-8807-fecc726f7218'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:41:28.826546 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-2ab0ba34-baac-4483-8807-fecc726f7218 admin admin] The token cache is being invalidate because project b36385de9bfd4ba691c28ab5550aa7dc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80311) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 01 16:41:28.827193 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-2ab0ba34-baac-4483-8807-fecc726f7218 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:41:28.828136 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-2ab0ba34-baac-4483-8807-fecc726f7218 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:41:28.830739 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1003/2022] 10.222.0.57 () {58 vars in 1232 bytes} [Fri Jul 1 16:41:28 2022] DELETE /identity/v3/projects/b36385de9bfd4ba691c28ab5550aa7dc => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 16:41:28.836278 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd0394b-ffc4-4f72-8155-9fb90c3201b9 None None] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:41:28.839386 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd0394b-ffc4-4f72-8155-9fb90c3201b9 None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:41:28.839750 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd0394b-ffc4-4f72-8155-9fb90c3201b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:41:28.864073 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.auth.core [None req-bbd0394b-ffc4-4f72-8155-9fb90c3201b9 None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80312) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:41:28.866610 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-bbd0394b-ffc4-4f72-8155-9fb90c3201b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:41:29.000206 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1020/2023] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 1 16:41:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 01 16:41:29.003518 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ac6387-d6bb-41f0-b0f0-e4e95cddfb47 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:41:29.003817 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ac6387-d6bb-41f0-b0f0-e4e95cddfb47 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:41:29.004097 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ac6387-d6bb-41f0-b0f0-e4e95cddfb47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:41:29.043646 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-d8ac6387-d6bb-41f0-b0f0-e4e95cddfb47 None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:41:29.044876 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-d8ac6387-d6bb-41f0-b0f0-e4e95cddfb47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:41:29.066330 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1004/2024] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 1 16:41:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 64 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 16:41:29.070335 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd0394b-ffc4-4f72-8155-9fb90c3201b9 None None] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:41:29.083899 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd0394b-ffc4-4f72-8155-9fb90c3201b9 None None] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:41:29.089336 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940fa0d1-c77b-4828-a2fa-6b312483f5b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:41:29.090141 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940fa0d1-c77b-4828-a2fa-6b312483f5b8 admin admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:41:29.090659 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940fa0d1-c77b-4828-a2fa-6b312483f5b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:41:29.091135 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940fa0d1-c77b-4828-a2fa-6b312483f5b8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:41:29.092160 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-940fa0d1-c77b-4828-a2fa-6b312483f5b8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80312) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:41:29.092581 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-940fa0d1-c77b-4828-a2fa-6b312483f5b8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:41:29.094286 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-940fa0d1-c77b-4828-a2fa-6b312483f5b8 admin admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:41:29.113509 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1021/2025] 10.222.0.57 () {58 vars in 1152 bytes} [Fri Jul 1 16:41:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 01 16:41:29.119003 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ac6387-d6bb-41f0-b0f0-e4e95cddfb47 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:41:29.133277 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ac6387-d6bb-41f0-b0f0-e4e95cddfb47 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:41:29.134420 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940fa0d1-c77b-4828-a2fa-6b312483f5b8 admin admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:41:29.136331 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454000c5-5bb9-490c-b323-022f85d4f75d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:41:29.136632 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-940fa0d1-c77b-4828-a2fa-6b312483f5b8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:41:29.137008 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454000c5-5bb9-490c-b323-022f85d4f75d admin admin] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:41:29.137300 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454000c5-5bb9-490c-b323-022f85d4f75d admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:41:29.137589 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454000c5-5bb9-490c-b323-022f85d4f75d admin admin] PATH_INFO: `/v3/projects` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:41:29.138537 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-454000c5-5bb9-490c-b323-022f85d4f75d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:41:29.140484 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-454000c5-5bb9-490c-b323-022f85d4f75d admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:41:29.164070 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940fa0d1-c77b-4828-a2fa-6b312483f5b8 admin admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:41:29.169305 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299f9c85-1ac5-4eac-9fbc-985d809fa4b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:41:29.170201 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299f9c85-1ac5-4eac-9fbc-985d809fa4b8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:41:29.170522 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299f9c85-1ac5-4eac-9fbc-985d809fa4b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:41:29.170812 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299f9c85-1ac5-4eac-9fbc-985d809fa4b8 admin admin] PATH_INFO: `/v3/users/8dbe1abec62c45fc941ffe30a4596a61` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:41:29.172029 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-299f9c85-1ac5-4eac-9fbc-985d809fa4b8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8dbe1abec62c45fc941ffe30a4596a61)` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:41:29.175915 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-299f9c85-1ac5-4eac-9fbc-985d809fa4b8 admin admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:41:29.216188 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1005/2026] 10.222.0.57 () {60 vars in 1152 bytes} [Fri Jul 1 16:41:29 2022] POST /identity/v3/projects => generated 367 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 16:41:29.220813 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454000c5-5bb9-490c-b323-022f85d4f75d admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:41:29.242939 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=84,keystone:DELETE=13,keystone:INSERT=3 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:41:29.262733 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-299f9c85-1ac5-4eac-9fbc-985d809fa4b8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8dbe1abec62c45fc941ffe30a4596a61'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:41:29.284737 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-299f9c85-1ac5-4eac-9fbc-985d809fa4b8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8dbe1abec62c45fc941ffe30a4596a61'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:41:29.297612 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-299f9c85-1ac5-4eac-9fbc-985d809fa4b8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8dbe1abec62c45fc941ffe30a4596a61'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:41:29.299214 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-299f9c85-1ac5-4eac-9fbc-985d809fa4b8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8dbe1abec62c45fc941ffe30a4596a61'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:41:29.299891 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-299f9c85-1ac5-4eac-9fbc-985d809fa4b8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8dbe1abec62c45fc941ffe30a4596a61'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:41:29.339523 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1022/2027] 10.222.0.57 () {58 vars in 1223 bytes} [Fri Jul 1 16:41:29 2022] DELETE /identity/v3/users/8dbe1abec62c45fc941ffe30a4596a61 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 16:41:29.344214 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d883edb6-ff2f-486a-aaec-0d2f0981d41e None None] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:41:29.344592 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d883edb6-ff2f-486a-aaec-0d2f0981d41e None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:41:29.345582 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d883edb6-ff2f-486a-aaec-0d2f0981d41e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:41:29.363347 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454000c5-5bb9-490c-b323-022f85d4f75d admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:41:29.372201 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.auth.core [None req-d883edb6-ff2f-486a-aaec-0d2f0981d41e None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80312) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:41:29.374510 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-d883edb6-ff2f-486a-aaec-0d2f0981d41e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:41:29.417498 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:DELETE=13,keystone:SELECT=85,keystone:INSERT=2 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:41:29.508202 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1023/2028] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 1 16:41:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 16:41:29.510463 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a91971a-1164-4c6e-8316-5dbbe9999410 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:41:29.511183 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a91971a-1164-4c6e-8316-5dbbe9999410 admin admin] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:41:29.511640 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a91971a-1164-4c6e-8316-5dbbe9999410 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:41:29.511934 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a91971a-1164-4c6e-8316-5dbbe9999410 admin admin] PATH_INFO: `/v3/users` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:41:29.512848 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a91971a-1164-4c6e-8316-5dbbe9999410 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:41:29.515181 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d883edb6-ff2f-486a-aaec-0d2f0981d41e None None] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:41:29.517653 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-d883edb6-ff2f-486a-aaec-0d2f0981d41e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:41:29.522026 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a91971a-1164-4c6e-8316-5dbbe9999410 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:41:29.555143 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1006/2029] 10.222.0.57 () {60 vars in 1143 bytes} [Fri Jul 1 16:41:29 2022] POST /identity/v3/users => generated 284 bytes in 335 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 01 16:41:29.566651 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a91971a-1164-4c6e-8316-5dbbe9999410 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:41:29.569197 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-4a91971a-1164-4c6e-8316-5dbbe9999410 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:41:29.593324 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a91971a-1164-4c6e-8316-5dbbe9999410 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:41:29.600581 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a823cb-552e-4e38-be8b-441220825129 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:41:29.601700 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a823cb-552e-4e38-be8b-441220825129 admin admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:41:29.602170 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a823cb-552e-4e38-be8b-441220825129 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:41:29.602663 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a823cb-552e-4e38-be8b-441220825129 admin admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:41:29.603771 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a823cb-552e-4e38-be8b-441220825129 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:41:29.605497 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a823cb-552e-4e38-be8b-441220825129 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:41:29.623009 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1007/2030] 10.222.0.57 () {58 vars in 1121 bytes} [Fri Jul 1 16:41:29 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 16:41:29.623205 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d883edb6-ff2f-486a-aaec-0d2f0981d41e None None] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:41:29.624109 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a823cb-552e-4e38-be8b-441220825129 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:41:29.638211 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a823cb-552e-4e38-be8b-441220825129 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:41:29.640806 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419e4abf-4f6c-479b-aa8a-6081b2aab898 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ed200ce29684c5f80354fa9a8638fe2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0788e6725a84e57a4174284c1845755', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:41:29.641519 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419e4abf-4f6c-479b-aa8a-6081b2aab898 service neutron] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:41:29.642667 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419e4abf-4f6c-479b-aa8a-6081b2aab898 service neutron] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:41:29.643424 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419e4abf-4f6c-479b-aa8a-6081b2aab898 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:41:29.651102 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8dc7bf-7997-43f0-9e3b-7bdb0d3a1027 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:41:29.651800 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8dc7bf-7997-43f0-9e3b-7bdb0d3a1027 admin admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:41:29.652090 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8dc7bf-7997-43f0-9e3b-7bdb0d3a1027 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:41:29.652372 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8dc7bf-7997-43f0-9e3b-7bdb0d3a1027 admin admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:41:29.653116 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f8dc7bf-7997-43f0-9e3b-7bdb0d3a1027 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:41:29.654776 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f8dc7bf-7997-43f0-9e3b-7bdb0d3a1027 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:41:29.667034 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1008/2031] 10.222.0.57 () {58 vars in 1121 bytes} [Fri Jul 1 16:41:29 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 01 16:41:29.669139 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-419e4abf-4f6c-479b-aa8a-6081b2aab898 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:41:29.670841 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-419e4abf-4f6c-479b-aa8a-6081b2aab898 service neutron] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:41:29.671975 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8dc7bf-7997-43f0-9e3b-7bdb0d3a1027 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:41:29.683718 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1024/2032] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 1 16:41:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 01 16:41:29.699814 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8dc7bf-7997-43f0-9e3b-7bdb0d3a1027 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:41:29.705518 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8b6a6f-f6f3-46f7-832d-5139519427aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:41:29.706540 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8b6a6f-f6f3-46f7-832d-5139519427aa admin admin] REQUEST_METHOD: `PUT` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:41:29.706831 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8b6a6f-f6f3-46f7-832d-5139519427aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:41:29.707177 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8b6a6f-f6f3-46f7-832d-5139519427aa admin admin] PATH_INFO: `/v3/projects/b33aef28a7e7420894bb07c1f2e60ad1/users/220054ce4e464f4da953bd16a4e0c5a0/roles/983be40bdab04c478eaa90c6b6a46b14` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:41:29.740837 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe8b6a6f-f6f3-46f7-832d-5139519427aa admin admin] RBAC: Authorizing `identity:create_grant(project_id=b33aef28a7e7420894bb07c1f2e60ad1, user_id=220054ce4e464f4da953bd16a4e0c5a0, role_id=983be40bdab04c478eaa90c6b6a46b14)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:41:29.743790 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe8b6a6f-f6f3-46f7-832d-5139519427aa admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:41:29.771685 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1009/2033] 10.222.0.57 () {60 vars in 1482 bytes} [Fri Jul 1 16:41:29 2022] PUT /identity/v3/projects/b33aef28a7e7420894bb07c1f2e60ad1/users/220054ce4e464f4da953bd16a4e0c5a0/roles/983be40bdab04c478eaa90c6b6a46b14 => generated 0 bytes in 92 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 16:41:29.774291 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b1e552-9d4a-4b16-bf1c-4848c3008827 None None] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:41:29.774623 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b1e552-9d4a-4b16-bf1c-4848c3008827 None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:41:29.774923 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b1e552-9d4a-4b16-bf1c-4848c3008827 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:41:29.809619 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.auth.core [None req-57b1e552-9d4a-4b16-bf1c-4848c3008827 None None] MFA Rules not processed for user `220054ce4e464f4da953bd16a4e0c5a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80312) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:41:29.811383 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-57b1e552-9d4a-4b16-bf1c-4848c3008827 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:41:29.914525 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1025/2034] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 1 16:41:29 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 16:41:29.919945 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8b6a6f-f6f3-46f7-832d-5139519427aa admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:41:30.094093 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8b6a6f-f6f3-46f7-832d-5139519427aa admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:41:30.106392 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5789de01-106d-4d05-acb4-0cff6a04d511 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:41:30.107068 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5789de01-106d-4d05-acb4-0cff6a04d511 admin admin] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:41:30.107355 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5789de01-106d-4d05-acb4-0cff6a04d511 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:41:30.107630 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5789de01-106d-4d05-acb4-0cff6a04d511 admin admin] PATH_INFO: `/v3/projects` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:41:30.108339 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5789de01-106d-4d05-acb4-0cff6a04d511 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:41:30.110788 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5789de01-106d-4d05-acb4-0cff6a04d511 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:41:30.120050 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:41:30.147838 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1010/2035] 10.222.0.57 () {60 vars in 1152 bytes} [Fri Jul 1 16:41:29 2022] POST /identity/v3/projects => generated 369 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 16:41:30.151514 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b1e552-9d4a-4b16-bf1c-4848c3008827 None None] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:41:30.164276 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b1e552-9d4a-4b16-bf1c-4848c3008827 None None] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:41:30.233416 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c43fa92-2b79-4179-b4ea-4c906f15dfc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:41:30.234013 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c43fa92-2b79-4179-b4ea-4c906f15dfc6 admin admin] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:41:30.234385 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c43fa92-2b79-4179-b4ea-4c906f15dfc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:41:30.234943 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c43fa92-2b79-4179-b4ea-4c906f15dfc6 admin admin] PATH_INFO: `/v3/users` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:41:30.235782 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c43fa92-2b79-4179-b4ea-4c906f15dfc6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:41:30.237644 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c43fa92-2b79-4179-b4ea-4c906f15dfc6 admin admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:41:30.249108 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:41:30.266804 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5789de01-106d-4d05-acb4-0cff6a04d511 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:41:30.269361 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1026/2036] 10.222.0.57 () {60 vars in 1143 bytes} [Fri Jul 1 16:41:30 2022] POST /identity/v3/users => generated 291 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 16:41:30.278445 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5789de01-106d-4d05-acb4-0cff6a04d511 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:41:30.284660 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf0894b-7421-4882-86cf-846483df9c3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:41:30.285527 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf0894b-7421-4882-86cf-846483df9c3e admin admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:41:30.285673 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf0894b-7421-4882-86cf-846483df9c3e admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:41:30.285894 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf0894b-7421-4882-86cf-846483df9c3e admin admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:41:30.286676 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf0894b-7421-4882-86cf-846483df9c3e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:41:30.288332 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf0894b-7421-4882-86cf-846483df9c3e admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:41:30.302199 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1011/2037] 10.222.0.57 () {58 vars in 1121 bytes} [Fri Jul 1 16:41:30 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 16:41:30.306064 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c43fa92-2b79-4179-b4ea-4c906f15dfc6 admin admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:41:30.317800 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c43fa92-2b79-4179-b4ea-4c906f15dfc6 admin admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:41:30.324213 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763d9c16-5dda-4842-b463-b2e36e9a562d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:41:30.325636 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763d9c16-5dda-4842-b463-b2e36e9a562d admin admin] REQUEST_METHOD: `PUT` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:41:30.326310 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763d9c16-5dda-4842-b463-b2e36e9a562d admin admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:41:30.326809 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763d9c16-5dda-4842-b463-b2e36e9a562d admin admin] PATH_INFO: `/v3/projects/e07de00e0b3a4983834b33d9a40ca306/users/0fe6265fb85e4499b4ea009486258251/roles/84ee739146934c1ab32c501ff8093644` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:41:30.346216 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-763d9c16-5dda-4842-b463-b2e36e9a562d admin admin] RBAC: Authorizing `identity:create_grant(project_id=e07de00e0b3a4983834b33d9a40ca306, user_id=0fe6265fb85e4499b4ea009486258251, role_id=84ee739146934c1ab32c501ff8093644)` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:41:30.351096 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-763d9c16-5dda-4842-b463-b2e36e9a562d admin admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:41:30.375632 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1027/2038] 10.222.0.57 () {60 vars in 1482 bytes} [Fri Jul 1 16:41:30 2022] PUT /identity/v3/projects/e07de00e0b3a4983834b33d9a40ca306/users/0fe6265fb85e4499b4ea009486258251/roles/84ee739146934c1ab32c501ff8093644 => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 16:41:30.380406 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf0894b-7421-4882-86cf-846483df9c3e admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:41:30.397542 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf0894b-7421-4882-86cf-846483df9c3e admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:41:30.507883 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58d91cc-9be7-4e4c-8555-e0af732b30b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:41:30.508862 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58d91cc-9be7-4e4c-8555-e0af732b30b0 admin admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:41:30.508862 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58d91cc-9be7-4e4c-8555-e0af732b30b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:41:30.509158 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58d91cc-9be7-4e4c-8555-e0af732b30b0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:41:30.509954 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d58d91cc-9be7-4e4c-8555-e0af732b30b0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:41:30.511865 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d58d91cc-9be7-4e4c-8555-e0af732b30b0 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:41:30.537484 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763d9c16-5dda-4842-b463-b2e36e9a562d admin admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:41:30.545733 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1012/2039] 10.222.0.57 () {58 vars in 1121 bytes} [Fri Jul 1 16:41:30 2022] GET /identity/v3/roles => generated 1420 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 01 16:41:30.552493 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763d9c16-5dda-4842-b463-b2e36e9a562d admin admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:41:30.558220 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59adc1e0-7d49-4870-8544-1f76fab13ecf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:41:30.559441 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59adc1e0-7d49-4870-8544-1f76fab13ecf admin admin] REQUEST_METHOD: `PUT` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:41:30.560082 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59adc1e0-7d49-4870-8544-1f76fab13ecf admin admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:41:30.560361 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59adc1e0-7d49-4870-8544-1f76fab13ecf admin admin] PATH_INFO: `/v3/projects/e07de00e0b3a4983834b33d9a40ca306/users/0fe6265fb85e4499b4ea009486258251/roles/84ee739146934c1ab32c501ff8093644` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:41:30.565247 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58d91cc-9be7-4e4c-8555-e0af732b30b0 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:41:30.567494 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-d58d91cc-9be7-4e4c-8555-e0af732b30b0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:41:30.577835 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59adc1e0-7d49-4870-8544-1f76fab13ecf admin admin] RBAC: Authorizing `identity:create_grant(project_id=e07de00e0b3a4983834b33d9a40ca306, user_id=0fe6265fb85e4499b4ea009486258251, role_id=84ee739146934c1ab32c501ff8093644)` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:41:30.599117 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59adc1e0-7d49-4870-8544-1f76fab13ecf admin admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:41:30.602691 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58d91cc-9be7-4e4c-8555-e0af732b30b0 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:41:30.618005 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80d58e5-69ec-4f46-b356-6d6cdfcc35d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:41:30.620446 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80d58e5-69ec-4f46-b356-6d6cdfcc35d8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:41:30.620988 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80d58e5-69ec-4f46-b356-6d6cdfcc35d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:41:30.621150 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80d58e5-69ec-4f46-b356-6d6cdfcc35d8 admin admin] PATH_INFO: `/v3/projects/9d29b085bfc54d3a94f6080bd0078690` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:41:30.624806 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c80d58e5-69ec-4f46-b356-6d6cdfcc35d8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9d29b085bfc54d3a94f6080bd0078690)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:41:30.627355 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c80d58e5-69ec-4f46-b356-6d6cdfcc35d8 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:41:30.640037 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1028/2040] 10.222.0.57 () {60 vars in 1482 bytes} [Fri Jul 1 16:41:30 2022] PUT /identity/v3/projects/e07de00e0b3a4983834b33d9a40ca306/users/0fe6265fb85e4499b4ea009486258251/roles/84ee739146934c1ab32c501ff8093644 => generated 0 bytes in 94 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 16:41:30.642766 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26cd1f2d-8adb-4cfc-b9ee-e5c9d347db63 None None] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:41:30.643154 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26cd1f2d-8adb-4cfc-b9ee-e5c9d347db63 None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:41:30.643629 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26cd1f2d-8adb-4cfc-b9ee-e5c9d347db63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:41:30.677037 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.auth.core [None req-26cd1f2d-8adb-4cfc-b9ee-e5c9d347db63 None None] MFA Rules not processed for user `0fe6265fb85e4499b4ea009486258251`. Rule list: `[]` (Enabled: `True`). {{(pid=80312) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:41:30.678396 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-26cd1f2d-8adb-4cfc-b9ee-e5c9d347db63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:41:30.717701 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-c80d58e5-69ec-4f46-b356-6d6cdfcc35d8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9d29b085bfc54d3a94f6080bd0078690', 'request_id': 'req-c80d58e5-69ec-4f46-b356-6d6cdfcc35d8'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:41:30.726892 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-c80d58e5-69ec-4f46-b356-6d6cdfcc35d8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9d29b085bfc54d3a94f6080bd0078690', 'request_id': 'req-c80d58e5-69ec-4f46-b356-6d6cdfcc35d8'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:41:30.742656 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-c80d58e5-69ec-4f46-b356-6d6cdfcc35d8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9d29b085bfc54d3a94f6080bd0078690', 'request_id': 'req-c80d58e5-69ec-4f46-b356-6d6cdfcc35d8'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:41:30.760978 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-c80d58e5-69ec-4f46-b356-6d6cdfcc35d8 admin admin] The token cache is being invalidate because project 9d29b085bfc54d3a94f6080bd0078690 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80311) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 01 16:41:30.761327 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-c80d58e5-69ec-4f46-b356-6d6cdfcc35d8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:41:30.762626 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-c80d58e5-69ec-4f46-b356-6d6cdfcc35d8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:41:30.764506 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1013/2041] 10.222.0.57 () {58 vars in 1232 bytes} [Fri Jul 1 16:41:30 2022] DELETE /identity/v3/projects/9d29b085bfc54d3a94f6080bd0078690 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 16:41:30.770069 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80d58e5-69ec-4f46-b356-6d6cdfcc35d8 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:41:30.772239 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-c80d58e5-69ec-4f46-b356-6d6cdfcc35d8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:41:30.777209 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1029/2042] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 1 16:41:30 2022] POST /identity/v3/auth/tokens => generated 3303 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 16:41:30.792465 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26cd1f2d-8adb-4cfc-b9ee-e5c9d347db63 None None] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:41:30.793878 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-26cd1f2d-8adb-4cfc-b9ee-e5c9d347db63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:41:30.861465 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:41:30.865681 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80d58e5-69ec-4f46-b356-6d6cdfcc35d8 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:41:30.873856 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26cd1f2d-8adb-4cfc-b9ee-e5c9d347db63 None None] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:41:30.874314 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462ba9e5-4a80-4d90-bb45-59c5bbd8cb73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:41:30.875244 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462ba9e5-4a80-4d90-bb45-59c5bbd8cb73 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:41:30.875526 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462ba9e5-4a80-4d90-bb45-59c5bbd8cb73 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:41:30.875798 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462ba9e5-4a80-4d90-bb45-59c5bbd8cb73 admin admin] PATH_INFO: `/v3/users/1409afa52a224eb18a51fdba56a04dff` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:41:30.877373 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-462ba9e5-4a80-4d90-bb45-59c5bbd8cb73 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1409afa52a224eb18a51fdba56a04dff)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:41:30.880397 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-462ba9e5-4a80-4d90-bb45-59c5bbd8cb73 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:41:30.931873 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894c6999-04ba-4dfa-b9c3-aa4cd6cae855 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbddf971a2354bbdbb9da1d63f3abf88', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0788e6725a84e57a4174284c1845755', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:41:30.932625 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894c6999-04ba-4dfa-b9c3-aa4cd6cae855 service cinder] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:41:30.932941 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894c6999-04ba-4dfa-b9c3-aa4cd6cae855 service cinder] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:41:30.933192 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894c6999-04ba-4dfa-b9c3-aa4cd6cae855 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:41:30.935038 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-894c6999-04ba-4dfa-b9c3-aa4cd6cae855 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:41:30.941240 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-462ba9e5-4a80-4d90-bb45-59c5bbd8cb73 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1409afa52a224eb18a51fdba56a04dff'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:41:30.942206 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-462ba9e5-4a80-4d90-bb45-59c5bbd8cb73 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1409afa52a224eb18a51fdba56a04dff'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:41:30.942880 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-462ba9e5-4a80-4d90-bb45-59c5bbd8cb73 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1409afa52a224eb18a51fdba56a04dff'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:41:30.946006 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:INSERT=4,keystone:SELECT=96 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:41:30.955308 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-462ba9e5-4a80-4d90-bb45-59c5bbd8cb73 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1409afa52a224eb18a51fdba56a04dff'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:41:30.969887 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-462ba9e5-4a80-4d90-bb45-59c5bbd8cb73 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1409afa52a224eb18a51fdba56a04dff'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:41:30.991858 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1014/2043] 10.222.0.57 () {58 vars in 1223 bytes} [Fri Jul 1 16:41:30 2022] DELETE /identity/v3/users/1409afa52a224eb18a51fdba56a04dff => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 01 16:41:30.991858 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5e0aaf-e416-443a-aaea-9e647118da3b None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:41:30.991858 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5e0aaf-e416-443a-aaea-9e647118da3b None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:41:30.991858 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5e0aaf-e416-443a-aaea-9e647118da3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:41:31.037808 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-894c6999-04ba-4dfa-b9c3-aa4cd6cae855 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:41:31.039794 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-894c6999-04ba-4dfa-b9c3-aa4cd6cae855 service cinder] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:41:31.051714 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1030/2044] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 1 16:41:30 2022] GET /identity/v3/auth/tokens => generated 3303 bytes in 260 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 01 16:41:31.082614 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-fe5e0aaf-e416-443a-aaea-9e647118da3b None None] MFA Rules not processed for user `c1ab0b52325c4ab387db034eb7dfcadb`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:41:31.084064 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-fe5e0aaf-e416-443a-aaea-9e647118da3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:41:31.200697 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1015/2045] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 1 16:41:30 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 16:41:31.206097 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894c6999-04ba-4dfa-b9c3-aa4cd6cae855 service cinder] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:41:31.207856 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-894c6999-04ba-4dfa-b9c3-aa4cd6cae855 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:41:31.253698 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894c6999-04ba-4dfa-b9c3-aa4cd6cae855 service cinder] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:41:31.375783 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a881f0b2-b024-43a9-8970-7b06a4e36a7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:41:31.376907 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a881f0b2-b024-43a9-8970-7b06a4e36a7a admin admin] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:41:31.377491 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a881f0b2-b024-43a9-8970-7b06a4e36a7a admin admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:41:31.378024 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a881f0b2-b024-43a9-8970-7b06a4e36a7a admin admin] PATH_INFO: `/v3/projects` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:41:31.379852 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a881f0b2-b024-43a9-8970-7b06a4e36a7a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:41:31.383422 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a881f0b2-b024-43a9-8970-7b06a4e36a7a admin admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:41:31.399951 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5e0aaf-e416-443a-aaea-9e647118da3b None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:41:31.401396 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-fe5e0aaf-e416-443a-aaea-9e647118da3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:41:31.438459 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5e0aaf-e416-443a-aaea-9e647118da3b None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:41:31.446834 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8224b8-61d4-4604-8d4c-3f5ac084c3f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:41:31.449682 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8224b8-61d4-4604-8d4c-3f5ac084c3f0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:41:31.449979 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8224b8-61d4-4604-8d4c-3f5ac084c3f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:41:31.450261 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8224b8-61d4-4604-8d4c-3f5ac084c3f0 admin admin] PATH_INFO: `/v3/projects/01c8d496300041168b3f96a865bcef1c` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:41:31.452364 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc8224b8-61d4-4604-8d4c-3f5ac084c3f0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=01c8d496300041168b3f96a865bcef1c)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:41:31.452781 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1031/2046] 10.222.0.57 () {60 vars in 1152 bytes} [Fri Jul 1 16:41:31 2022] POST /identity/v3/projects => generated 385 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 16:41:31.455198 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc8224b8-61d4-4604-8d4c-3f5ac084c3f0 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:41:31.458187 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a881f0b2-b024-43a9-8970-7b06a4e36a7a admin admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:41:31.460768 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-a881f0b2-b024-43a9-8970-7b06a4e36a7a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:41:31.550058 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-dc8224b8-61d4-4604-8d4c-3f5ac084c3f0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '01c8d496300041168b3f96a865bcef1c', 'request_id': 'req-dc8224b8-61d4-4604-8d4c-3f5ac084c3f0'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:41:31.556012 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-dc8224b8-61d4-4604-8d4c-3f5ac084c3f0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '01c8d496300041168b3f96a865bcef1c', 'request_id': 'req-dc8224b8-61d4-4604-8d4c-3f5ac084c3f0'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:41:31.558810 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:41:31.560697 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a881f0b2-b024-43a9-8970-7b06a4e36a7a admin admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:41:31.567074 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-dc8224b8-61d4-4604-8d4c-3f5ac084c3f0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '01c8d496300041168b3f96a865bcef1c', 'request_id': 'req-dc8224b8-61d4-4604-8d4c-3f5ac084c3f0'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:41:31.588778 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-dc8224b8-61d4-4604-8d4c-3f5ac084c3f0 admin admin] The token cache is being invalidate because project 01c8d496300041168b3f96a865bcef1c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80311) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 01 16:41:31.589143 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-dc8224b8-61d4-4604-8d4c-3f5ac084c3f0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:41:31.589843 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-dc8224b8-61d4-4604-8d4c-3f5ac084c3f0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:41:31.592251 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1016/2047] 10.222.0.57 () {58 vars in 1232 bytes} [Fri Jul 1 16:41:31 2022] DELETE /identity/v3/projects/01c8d496300041168b3f96a865bcef1c => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 16:41:31.598989 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0caf362a-bf01-4575-b7f7-6cf6a6a00755 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:41:31.599505 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0caf362a-bf01-4575-b7f7-6cf6a6a00755 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:41:31.599972 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0caf362a-bf01-4575-b7f7-6cf6a6a00755 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:41:31.604566 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:41:31.620979 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-0caf362a-bf01-4575-b7f7-6cf6a6a00755 None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:41:31.622200 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-0caf362a-bf01-4575-b7f7-6cf6a6a00755 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:41:31.636995 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b67e452-c59b-4571-b730-99e495c5fc83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:41:31.637923 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b67e452-c59b-4571-b730-99e495c5fc83 admin admin] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:41:31.638213 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b67e452-c59b-4571-b730-99e495c5fc83 admin admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:41:31.638573 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b67e452-c59b-4571-b730-99e495c5fc83 admin admin] PATH_INFO: `/v3/users` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:41:31.639312 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b67e452-c59b-4571-b730-99e495c5fc83 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:41:31.641449 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b67e452-c59b-4571-b730-99e495c5fc83 admin admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:41:31.669901 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1032/2048] 10.222.0.57 () {60 vars in 1143 bytes} [Fri Jul 1 16:41:31 2022] POST /identity/v3/users => generated 299 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 01 16:41:31.670177 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b67e452-c59b-4571-b730-99e495c5fc83 admin admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:41:31.671888 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-8b67e452-c59b-4571-b730-99e495c5fc83 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:41:31.691533 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b67e452-c59b-4571-b730-99e495c5fc83 admin admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:41:31.697109 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7716a357-0ff2-469d-85d8-b673f35a4b4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:41:31.697792 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7716a357-0ff2-469d-85d8-b673f35a4b4e admin admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:41:31.698079 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7716a357-0ff2-469d-85d8-b673f35a4b4e admin admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:41:31.698374 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7716a357-0ff2-469d-85d8-b673f35a4b4e admin admin] PATH_INFO: `/v3/roles` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:41:31.699294 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7716a357-0ff2-469d-85d8-b673f35a4b4e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:41:31.701077 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7716a357-0ff2-469d-85d8-b673f35a4b4e admin admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:41:31.710800 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1017/2049] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 1 16:41:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 16:41:31.715251 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1033/2050] 10.222.0.57 () {58 vars in 1121 bytes} [Fri Jul 1 16:41:31 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 01 16:41:31.717692 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0caf362a-bf01-4575-b7f7-6cf6a6a00755 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:41:31.718175 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd001dd5-85a0-4946-bee8-04b272726e3b None None] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:41:31.718502 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd001dd5-85a0-4946-bee8-04b272726e3b None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:41:31.718787 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd001dd5-85a0-4946-bee8-04b272726e3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:41:31.729530 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0caf362a-bf01-4575-b7f7-6cf6a6a00755 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:41:31.738447 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601238cf-51a5-47fe-985a-144ee667b8fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:41:31.739582 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601238cf-51a5-47fe-985a-144ee667b8fc admin admin] REQUEST_METHOD: `PUT` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:41:31.739701 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.auth.core [None req-fd001dd5-85a0-4946-bee8-04b272726e3b None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80312) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:41:31.740134 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601238cf-51a5-47fe-985a-144ee667b8fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:41:31.740713 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601238cf-51a5-47fe-985a-144ee667b8fc admin admin] PATH_INFO: `/v3/projects/f8d3731c3718400b9f8143151dfaac9f/users/4a269ade40e44ad986f32ce794371e8b/roles/84ee739146934c1ab32c501ff8093644` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:41:31.740885 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-fd001dd5-85a0-4946-bee8-04b272726e3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:41:31.754175 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1034/2051] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 1 16:41:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 16:41:31.755253 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd001dd5-85a0-4946-bee8-04b272726e3b None None] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:41:31.764923 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd001dd5-85a0-4946-bee8-04b272726e3b None None] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:41:31.772234 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203d7b87-6b59-41a6-86d0-c9a50537685f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:41:31.772951 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203d7b87-6b59-41a6-86d0-c9a50537685f admin admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:41:31.773580 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203d7b87-6b59-41a6-86d0-c9a50537685f admin admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:41:31.773580 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203d7b87-6b59-41a6-86d0-c9a50537685f admin admin] PATH_INFO: `/v3/domains` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:41:31.774590 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-203d7b87-6b59-41a6-86d0-c9a50537685f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80312) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:41:31.774590 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-203d7b87-6b59-41a6-86d0-c9a50537685f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:41:31.779324 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-601238cf-51a5-47fe-985a-144ee667b8fc admin admin] RBAC: Authorizing `identity:create_grant(project_id=f8d3731c3718400b9f8143151dfaac9f, user_id=4a269ade40e44ad986f32ce794371e8b, role_id=84ee739146934c1ab32c501ff8093644)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:41:31.781252 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-601238cf-51a5-47fe-985a-144ee667b8fc admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:41:31.786719 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-203d7b87-6b59-41a6-86d0-c9a50537685f admin admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:41:31.802121 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1035/2052] 10.222.0.57 () {58 vars in 1152 bytes} [Fri Jul 1 16:41:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:41:31.803071 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1018/2053] 10.222.0.57 () {60 vars in 1482 bytes} [Fri Jul 1 16:41:31 2022] PUT /identity/v3/projects/f8d3731c3718400b9f8143151dfaac9f/users/4a269ade40e44ad986f32ce794371e8b/roles/84ee739146934c1ab32c501ff8093644 => generated 0 bytes in 86 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 16:41:31.807385 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203d7b87-6b59-41a6-86d0-c9a50537685f admin admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:41:31.807894 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601238cf-51a5-47fe-985a-144ee667b8fc admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:41:31.819536 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203d7b87-6b59-41a6-86d0-c9a50537685f admin admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:41:31.904895 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a5f0b7-6fdb-4e36-8bb4-41639effd793 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:41:31.906080 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a5f0b7-6fdb-4e36-8bb4-41639effd793 admin admin] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:41:31.906439 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a5f0b7-6fdb-4e36-8bb4-41639effd793 admin admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:41:31.906929 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a5f0b7-6fdb-4e36-8bb4-41639effd793 admin admin] PATH_INFO: `/v3/projects` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:41:31.908267 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a5f0b7-6fdb-4e36-8bb4-41639effd793 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:41:31.911035 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a5f0b7-6fdb-4e36-8bb4-41639effd793 admin admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:41:31.921805 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601238cf-51a5-47fe-985a-144ee667b8fc admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:41:31.927561 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c6d54d-0001-475a-92a5-e26e23fac127 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:41:31.928530 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c6d54d-0001-475a-92a5-e26e23fac127 admin admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:41:31.928880 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c6d54d-0001-475a-92a5-e26e23fac127 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:41:31.929230 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c6d54d-0001-475a-92a5-e26e23fac127 admin admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:41:31.930405 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c6d54d-0001-475a-92a5-e26e23fac127 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:41:31.932339 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c6d54d-0001-475a-92a5-e26e23fac127 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:41:31.945046 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1019/2054] 10.222.0.57 () {58 vars in 1121 bytes} [Fri Jul 1 16:41:31 2022] GET /identity/v3/roles => generated 1420 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 16:41:31.949473 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c6d54d-0001-475a-92a5-e26e23fac127 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:41:31.956581 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1036/2055] 10.222.0.57 () {60 vars in 1152 bytes} [Fri Jul 1 16:41:31 2022] POST /identity/v3/projects => generated 365 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 16:41:31.957864 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a5f0b7-6fdb-4e36-8bb4-41639effd793 admin admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:41:31.963459 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c6d54d-0001-475a-92a5-e26e23fac127 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:41:32.050144 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb70fe5e-023d-4e1d-aed0-37adc23d1c54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:41:32.051624 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb70fe5e-023d-4e1d-aed0-37adc23d1c54 admin admin] REQUEST_METHOD: `PUT` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:41:32.052087 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb70fe5e-023d-4e1d-aed0-37adc23d1c54 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:41:32.052636 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb70fe5e-023d-4e1d-aed0-37adc23d1c54 admin admin] PATH_INFO: `/v3/projects/f8d3731c3718400b9f8143151dfaac9f/users/4a269ade40e44ad986f32ce794371e8b/roles/84ee739146934c1ab32c501ff8093644` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:41:32.056652 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a5f0b7-6fdb-4e36-8bb4-41639effd793 admin admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:41:32.058396 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb70fe5e-023d-4e1d-aed0-37adc23d1c54 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f8d3731c3718400b9f8143151dfaac9f, user_id=4a269ade40e44ad986f32ce794371e8b, role_id=84ee739146934c1ab32c501ff8093644)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:41:32.061394 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb70fe5e-023d-4e1d-aed0-37adc23d1c54 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:41:32.063578 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728bf0f1-2806-4a71-97b0-af0d1a356f8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:41:32.065865 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=96,keystone:INSERT=3,keystone:DELETE=1 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:41:32.066087 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728bf0f1-2806-4a71-97b0-af0d1a356f8a admin admin] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:41:32.067654 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728bf0f1-2806-4a71-97b0-af0d1a356f8a admin admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:41:32.068110 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728bf0f1-2806-4a71-97b0-af0d1a356f8a admin admin] PATH_INFO: `/v3/users` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:41:32.069863 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-728bf0f1-2806-4a71-97b0-af0d1a356f8a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:41:32.074663 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-728bf0f1-2806-4a71-97b0-af0d1a356f8a admin admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:41:32.081965 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1020/2056] 10.222.0.57 () {60 vars in 1482 bytes} [Fri Jul 1 16:41:31 2022] PUT /identity/v3/projects/f8d3731c3718400b9f8143151dfaac9f/users/4a269ade40e44ad986f32ce794371e8b/roles/84ee739146934c1ab32c501ff8093644 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 16:41:32.106407 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1037/2057] 10.222.0.57 () {60 vars in 1143 bytes} [Fri Jul 1 16:41:31 2022] POST /identity/v3/users => generated 283 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 16:41:32.111388 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb70fe5e-023d-4e1d-aed0-37adc23d1c54 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:41:32.209930 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb70fe5e-023d-4e1d-aed0-37adc23d1c54 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:41:32.225952 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70fd15ad-a85b-4e62-a9ad-6f96d381e5e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:41:32.227852 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70fd15ad-a85b-4e62-a9ad-6f96d381e5e7 admin admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:41:32.227852 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70fd15ad-a85b-4e62-a9ad-6f96d381e5e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:41:32.227852 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70fd15ad-a85b-4e62-a9ad-6f96d381e5e7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:41:32.228172 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70fd15ad-a85b-4e62-a9ad-6f96d381e5e7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:41:32.229701 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70fd15ad-a85b-4e62-a9ad-6f96d381e5e7 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:41:32.244521 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1021/2058] 10.222.0.57 () {58 vars in 1121 bytes} [Fri Jul 1 16:41:32 2022] GET /identity/v3/roles => generated 1420 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 16:41:32.248507 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728bf0f1-2806-4a71-97b0-af0d1a356f8a admin admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:41:32.276578 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728bf0f1-2806-4a71-97b0-af0d1a356f8a admin admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:41:32.297564 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9f5efd-4cf2-4c33-94e4-75ae77848be7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:41:32.298427 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9f5efd-4cf2-4c33-94e4-75ae77848be7 admin admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:41:32.298782 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9f5efd-4cf2-4c33-94e4-75ae77848be7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:41:32.299064 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9f5efd-4cf2-4c33-94e4-75ae77848be7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:41:32.299927 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db9f5efd-4cf2-4c33-94e4-75ae77848be7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:41:32.301682 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db9f5efd-4cf2-4c33-94e4-75ae77848be7 admin admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:41:32.315310 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:41:32.324136 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1038/2059] 10.222.0.57 () {58 vars in 1121 bytes} [Fri Jul 1 16:41:32 2022] GET /identity/v3/roles => generated 1420 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 16:41:32.328315 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70fd15ad-a85b-4e62-a9ad-6f96d381e5e7 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:41:32.345584 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70fd15ad-a85b-4e62-a9ad-6f96d381e5e7 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:41:32.370488 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eddf49e6-b993-4941-817a-d70b1070f23b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:41:32.373891 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddf49e6-b993-4941-817a-d70b1070f23b admin admin] REQUEST_METHOD: `PUT` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:41:32.374218 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddf49e6-b993-4941-817a-d70b1070f23b admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:41:32.374533 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddf49e6-b993-4941-817a-d70b1070f23b admin admin] PATH_INFO: `/v3/projects/6ee973569cba400f813060ab544d4eab/users/25f8caac070f4d3daa1777fe145432a4/roles/983be40bdab04c478eaa90c6b6a46b14` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:41:32.411504 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eddf49e6-b993-4941-817a-d70b1070f23b admin admin] RBAC: Authorizing `identity:create_grant(project_id=6ee973569cba400f813060ab544d4eab, user_id=25f8caac070f4d3daa1777fe145432a4, role_id=983be40bdab04c478eaa90c6b6a46b14)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:41:32.427476 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eddf49e6-b993-4941-817a-d70b1070f23b admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:41:32.448930 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1022/2060] 10.222.0.57 () {60 vars in 1482 bytes} [Fri Jul 1 16:41:32 2022] PUT /identity/v3/projects/6ee973569cba400f813060ab544d4eab/users/25f8caac070f4d3daa1777fe145432a4/roles/983be40bdab04c478eaa90c6b6a46b14 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 16:41:32.458281 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9e1a19-8427-4528-93f9-7c93bb9c0ac9 None None] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:41:32.458632 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9e1a19-8427-4528-93f9-7c93bb9c0ac9 None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:41:32.458968 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9e1a19-8427-4528-93f9-7c93bb9c0ac9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:41:32.503004 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.auth.core [None req-bf9e1a19-8427-4528-93f9-7c93bb9c0ac9 None None] MFA Rules not processed for user `25f8caac070f4d3daa1777fe145432a4`. Rule list: `[]` (Enabled: `True`). {{(pid=80312) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:41:32.505003 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-bf9e1a19-8427-4528-93f9-7c93bb9c0ac9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:41:32.641537 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1039/2061] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 1 16:41:32 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 16:41:32.648633 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eddf49e6-b993-4941-817a-d70b1070f23b admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:41:32.748694 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eddf49e6-b993-4941-817a-d70b1070f23b admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:41:32.763731 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f051092-a0d2-47d3-9131-ac2a970f49e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:41:32.765142 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f051092-a0d2-47d3-9131-ac2a970f49e7 admin admin] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:41:32.765525 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f051092-a0d2-47d3-9131-ac2a970f49e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:41:32.770962 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f051092-a0d2-47d3-9131-ac2a970f49e7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:41:32.771840 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f051092-a0d2-47d3-9131-ac2a970f49e7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:41:32.775540 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f051092-a0d2-47d3-9131-ac2a970f49e7 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:41:32.814534 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1023/2062] 10.222.0.57 () {60 vars in 1152 bytes} [Fri Jul 1 16:41:32 2022] POST /identity/v3/projects => generated 365 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 16:41:32.816365 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9e1a19-8427-4528-93f9-7c93bb9c0ac9 None None] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:41:32.830611 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9e1a19-8427-4528-93f9-7c93bb9c0ac9 None None] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:41:33.011317 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0084dfab-18da-4c0c-93d2-dcb4ce83a426 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:41:33.012376 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0084dfab-18da-4c0c-93d2-dcb4ce83a426 admin admin] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:41:33.014476 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0084dfab-18da-4c0c-93d2-dcb4ce83a426 admin admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:41:33.015349 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0084dfab-18da-4c0c-93d2-dcb4ce83a426 admin admin] PATH_INFO: `/v3/users` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:41:33.018213 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0084dfab-18da-4c0c-93d2-dcb4ce83a426 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:41:33.025336 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0084dfab-18da-4c0c-93d2-dcb4ce83a426 admin admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:41:33.100009 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1040/2063] 10.222.0.57 () {60 vars in 1143 bytes} [Fri Jul 1 16:41:32 2022] POST /identity/v3/users => generated 289 bytes in 283 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 16:41:33.103513 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f051092-a0d2-47d3-9131-ac2a970f49e7 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:41:33.117920 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f051092-a0d2-47d3-9131-ac2a970f49e7 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:41:33.133631 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14463545-f6c3-4e7a-9516-7cacba9bf773 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:41:33.134339 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14463545-f6c3-4e7a-9516-7cacba9bf773 admin admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:41:33.134671 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14463545-f6c3-4e7a-9516-7cacba9bf773 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:41:33.134981 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14463545-f6c3-4e7a-9516-7cacba9bf773 admin admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:41:33.136246 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14463545-f6c3-4e7a-9516-7cacba9bf773 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:41:33.137972 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14463545-f6c3-4e7a-9516-7cacba9bf773 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:41:33.152150 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1024/2064] 10.222.0.57 () {58 vars in 1121 bytes} [Fri Jul 1 16:41:33 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 16:41:33.156708 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0084dfab-18da-4c0c-93d2-dcb4ce83a426 admin admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:41:33.169097 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0084dfab-18da-4c0c-93d2-dcb4ce83a426 admin admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:41:33.177352 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aadc85f-11e5-4f25-8766-7cbca0ff276a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:41:33.178315 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aadc85f-11e5-4f25-8766-7cbca0ff276a admin admin] REQUEST_METHOD: `PUT` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:41:33.178812 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aadc85f-11e5-4f25-8766-7cbca0ff276a admin admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:41:33.179075 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aadc85f-11e5-4f25-8766-7cbca0ff276a admin admin] PATH_INFO: `/v3/projects/947c7f038ac44414a96a2f273cd74437/users/21d2f2ba617a41bd9d4640f8a7f5488c/roles/84ee739146934c1ab32c501ff8093644` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:41:33.200476 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aadc85f-11e5-4f25-8766-7cbca0ff276a admin admin] RBAC: Authorizing `identity:create_grant(project_id=947c7f038ac44414a96a2f273cd74437, user_id=21d2f2ba617a41bd9d4640f8a7f5488c, role_id=84ee739146934c1ab32c501ff8093644)` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:41:33.205373 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aadc85f-11e5-4f25-8766-7cbca0ff276a admin admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:41:33.240822 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1041/2065] 10.222.0.57 () {60 vars in 1482 bytes} [Fri Jul 1 16:41:33 2022] PUT /identity/v3/projects/947c7f038ac44414a96a2f273cd74437/users/21d2f2ba617a41bd9d4640f8a7f5488c/roles/84ee739146934c1ab32c501ff8093644 => generated 0 bytes in 85 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 16:41:33.244980 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14463545-f6c3-4e7a-9516-7cacba9bf773 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:41:33.261766 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14463545-f6c3-4e7a-9516-7cacba9bf773 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:41:33.270530 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aadc85f-11e5-4f25-8766-7cbca0ff276a admin admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:41:33.272429 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-3aadc85f-11e5-4f25-8766-7cbca0ff276a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:41:33.324108 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:41:33.349547 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f91761-8b11-4404-94a7-fc09396cfa23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:41:33.350861 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f91761-8b11-4404-94a7-fc09396cfa23 admin admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:41:33.351080 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f91761-8b11-4404-94a7-fc09396cfa23 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:41:33.351511 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f91761-8b11-4404-94a7-fc09396cfa23 admin admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:41:33.352527 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f91761-8b11-4404-94a7-fc09396cfa23 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:41:33.354639 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f91761-8b11-4404-94a7-fc09396cfa23 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:41:33.360060 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:41:33.370174 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aadc85f-11e5-4f25-8766-7cbca0ff276a admin admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:41:33.373667 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1025/2066] 10.222.0.57 () {58 vars in 1121 bytes} [Fri Jul 1 16:41:33 2022] GET /identity/v3/roles => generated 1420 bytes in 129 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 16:41:33.375289 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9be185-0743-4701-a33e-51d1882a8b96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:41:33.375797 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f91761-8b11-4404-94a7-fc09396cfa23 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:41:33.377211 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9be185-0743-4701-a33e-51d1882a8b96 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:41:33.378081 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9be185-0743-4701-a33e-51d1882a8b96 admin admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:41:33.378856 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9be185-0743-4701-a33e-51d1882a8b96 admin admin] PATH_INFO: `/v3/users/220054ce4e464f4da953bd16a4e0c5a0` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:41:33.380898 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f9be185-0743-4701-a33e-51d1882a8b96 admin admin] RBAC: Authorizing `identity:delete_user(user_id=220054ce4e464f4da953bd16a4e0c5a0)` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:41:33.383157 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f9be185-0743-4701-a33e-51d1882a8b96 admin admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:41:33.390276 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f91761-8b11-4404-94a7-fc09396cfa23 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:41:33.403302 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe17f2c-382e-4526-8e48-21f8a37cdc24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:41:33.404272 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe17f2c-382e-4526-8e48-21f8a37cdc24 admin admin] REQUEST_METHOD: `PUT` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:41:33.404561 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe17f2c-382e-4526-8e48-21f8a37cdc24 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:41:33.404839 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe17f2c-382e-4526-8e48-21f8a37cdc24 admin admin] PATH_INFO: `/v3/projects/947c7f038ac44414a96a2f273cd74437/users/21d2f2ba617a41bd9d4640f8a7f5488c/roles/84ee739146934c1ab32c501ff8093644` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:41:33.409470 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe17f2c-382e-4526-8e48-21f8a37cdc24 admin admin] RBAC: Authorizing `identity:create_grant(project_id=947c7f038ac44414a96a2f273cd74437, user_id=21d2f2ba617a41bd9d4640f8a7f5488c, role_id=84ee739146934c1ab32c501ff8093644)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:41:33.412597 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe17f2c-382e-4526-8e48-21f8a37cdc24 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:41:33.433311 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1026/2067] 10.222.0.57 () {60 vars in 1482 bytes} [Fri Jul 1 16:41:33 2022] PUT /identity/v3/projects/947c7f038ac44414a96a2f273cd74437/users/21d2f2ba617a41bd9d4640f8a7f5488c/roles/84ee739146934c1ab32c501ff8093644 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 16:41:33.435095 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cece9e0a-5da9-4e6b-86f1-b25acc9d69be None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:41:33.435388 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cece9e0a-5da9-4e6b-86f1-b25acc9d69be None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:41:33.435673 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cece9e0a-5da9-4e6b-86f1-b25acc9d69be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:41:33.479850 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-cece9e0a-5da9-4e6b-86f1-b25acc9d69be None None] MFA Rules not processed for user `21d2f2ba617a41bd9d4640f8a7f5488c`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:41:33.481664 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-cece9e0a-5da9-4e6b-86f1-b25acc9d69be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:41:33.503621 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-3f9be185-0743-4701-a33e-51d1882a8b96 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '220054ce4e464f4da953bd16a4e0c5a0'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:41:33.538444 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-3f9be185-0743-4701-a33e-51d1882a8b96 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '220054ce4e464f4da953bd16a4e0c5a0'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:41:33.564836 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-3f9be185-0743-4701-a33e-51d1882a8b96 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '220054ce4e464f4da953bd16a4e0c5a0'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:41:33.566609 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-3f9be185-0743-4701-a33e-51d1882a8b96 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '220054ce4e464f4da953bd16a4e0c5a0'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:41:33.567880 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-3f9be185-0743-4701-a33e-51d1882a8b96 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '220054ce4e464f4da953bd16a4e0c5a0'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:41:33.597456 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1042/2068] 10.222.0.57 () {58 vars in 1223 bytes} [Fri Jul 1 16:41:33 2022] DELETE /identity/v3/users/220054ce4e464f4da953bd16a4e0c5a0 => generated 0 bytes in 327 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 01 16:41:33.602581 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ff835d-8685-446c-8ddd-53eb39ede433 None None] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:41:33.604533 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ff835d-8685-446c-8ddd-53eb39ede433 None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:41:33.605692 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ff835d-8685-446c-8ddd-53eb39ede433 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:41:33.651160 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.auth.core [None req-a1ff835d-8685-446c-8ddd-53eb39ede433 None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80312) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:41:33.653654 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-a1ff835d-8685-446c-8ddd-53eb39ede433 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:41:33.677710 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1027/2069] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 1 16:41:33 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 243 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 01 16:41:33.690263 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cece9e0a-5da9-4e6b-86f1-b25acc9d69be None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:41:33.691684 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-cece9e0a-5da9-4e6b-86f1-b25acc9d69be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:41:33.807927 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1043/2070] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 1 16:41:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 16:41:33.818596 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ff835d-8685-446c-8ddd-53eb39ede433 None None] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:41:33.820344 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-a1ff835d-8685-446c-8ddd-53eb39ede433 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:41:33.831165 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cece9e0a-5da9-4e6b-86f1-b25acc9d69be None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:41:33.956094 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37acc3f-ad87-4ed5-b5ab-6560399d1515 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbddf971a2354bbdbb9da1d63f3abf88', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0788e6725a84e57a4174284c1845755', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:41:33.957262 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37acc3f-ad87-4ed5-b5ab-6560399d1515 service cinder] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:41:33.957551 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37acc3f-ad87-4ed5-b5ab-6560399d1515 service cinder] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:41:33.957831 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37acc3f-ad87-4ed5-b5ab-6560399d1515 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:41:33.996850 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ff835d-8685-446c-8ddd-53eb39ede433 None None] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:41:34.002948 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:41:34.004234 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca69270e-59bc-4e9b-bb6c-3eafdba80192 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ed200ce29684c5f80354fa9a8638fe2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0788e6725a84e57a4174284c1845755', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:41:34.005452 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca69270e-59bc-4e9b-bb6c-3eafdba80192 service neutron] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:41:34.006751 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca69270e-59bc-4e9b-bb6c-3eafdba80192 service neutron] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:41:34.007246 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca69270e-59bc-4e9b-bb6c-3eafdba80192 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:41:34.021005 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca69270e-59bc-4e9b-bb6c-3eafdba80192 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:41:34.023080 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca69270e-59bc-4e9b-bb6c-3eafdba80192 service neutron] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:41:34.042039 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1044/2071] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 1 16:41:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 224 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 16:41:34.056929 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f37acc3f-ad87-4ed5-b5ab-6560399d1515 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:41:34.058488 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f37acc3f-ad87-4ed5-b5ab-6560399d1515 service cinder] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:41:34.069789 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1028/2072] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 1 16:41:33 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 16:41:34.211900 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca69270e-59bc-4e9b-bb6c-3eafdba80192 service neutron] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:41:34.226244 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca69270e-59bc-4e9b-bb6c-3eafdba80192 service neutron] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:41:34.242165 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ecf290-7971-4ec1-bb39-80ea063b4bed service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbddf971a2354bbdbb9da1d63f3abf88', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0788e6725a84e57a4174284c1845755', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:41:34.243056 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ecf290-7971-4ec1-bb39-80ea063b4bed service cinder] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:41:34.243239 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ecf290-7971-4ec1-bb39-80ea063b4bed service cinder] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:41:34.243475 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ecf290-7971-4ec1-bb39-80ea063b4bed service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:41:34.250454 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-b6ecf290-7971-4ec1-bb39-80ea063b4bed service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:41:34.282121 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:41:34.374475 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ecf290-7971-4ec1-bb39-80ea063b4bed service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:41:34.376128 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ecf290-7971-4ec1-bb39-80ea063b4bed service cinder] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:41:34.388356 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1045/2073] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 1 16:41:34 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 180 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 16:41:34.630098 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37acc3f-ad87-4ed5-b5ab-6560399d1515 service cinder] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:41:34.631891 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-f37acc3f-ad87-4ed5-b5ab-6560399d1515 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:41:34.664090 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37acc3f-ad87-4ed5-b5ab-6560399d1515 service cinder] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:41:34.682454 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb396fa-c523-44ce-af63-9fa1256cec6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:41:34.683959 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb396fa-c523-44ce-af63-9fa1256cec6e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:41:34.684453 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb396fa-c523-44ce-af63-9fa1256cec6e admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:41:34.684915 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb396fa-c523-44ce-af63-9fa1256cec6e admin admin] PATH_INFO: `/v3/projects/b33aef28a7e7420894bb07c1f2e60ad1` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:41:34.691052 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb396fa-c523-44ce-af63-9fa1256cec6e admin admin] RBAC: Authorizing `identity:delete_project(project_id=b33aef28a7e7420894bb07c1f2e60ad1)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:41:34.693479 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb396fa-c523-44ce-af63-9fa1256cec6e admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:41:34.805435 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-fbb396fa-c523-44ce-af63-9fa1256cec6e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b33aef28a7e7420894bb07c1f2e60ad1', 'request_id': 'req-fbb396fa-c523-44ce-af63-9fa1256cec6e'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:41:34.817352 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-fbb396fa-c523-44ce-af63-9fa1256cec6e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b33aef28a7e7420894bb07c1f2e60ad1', 'request_id': 'req-fbb396fa-c523-44ce-af63-9fa1256cec6e'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:41:34.825660 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-fbb396fa-c523-44ce-af63-9fa1256cec6e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b33aef28a7e7420894bb07c1f2e60ad1', 'request_id': 'req-fbb396fa-c523-44ce-af63-9fa1256cec6e'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:41:34.848551 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-fbb396fa-c523-44ce-af63-9fa1256cec6e admin admin] The token cache is being invalidate because project b33aef28a7e7420894bb07c1f2e60ad1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80311) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 01 16:41:34.849359 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-fbb396fa-c523-44ce-af63-9fa1256cec6e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:41:34.850225 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-fbb396fa-c523-44ce-af63-9fa1256cec6e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:41:34.856495 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1029/2074] 10.222.0.57 () {58 vars in 1232 bytes} [Fri Jul 1 16:41:34 2022] DELETE /identity/v3/projects/b33aef28a7e7420894bb07c1f2e60ad1 => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 16:41:34.860298 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ecf290-7971-4ec1-bb39-80ea063b4bed service cinder] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:41:34.876902 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ecf290-7971-4ec1-bb39-80ea063b4bed service cinder] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:41:34.980521 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241740ac-223f-4e99-a822-5ea8fe5c2064 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:41:34.981533 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241740ac-223f-4e99-a822-5ea8fe5c2064 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:41:34.981960 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241740ac-223f-4e99-a822-5ea8fe5c2064 admin admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:41:34.982244 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241740ac-223f-4e99-a822-5ea8fe5c2064 admin admin] PATH_INFO: `/v3/users/0fe6265fb85e4499b4ea009486258251` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:41:34.984362 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-241740ac-223f-4e99-a822-5ea8fe5c2064 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0fe6265fb85e4499b4ea009486258251)` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:41:34.989567 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-241740ac-223f-4e99-a822-5ea8fe5c2064 admin admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:41:35.083957 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-241740ac-223f-4e99-a822-5ea8fe5c2064 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0fe6265fb85e4499b4ea009486258251'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:41:35.129783 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-241740ac-223f-4e99-a822-5ea8fe5c2064 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0fe6265fb85e4499b4ea009486258251'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:41:35.154319 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-241740ac-223f-4e99-a822-5ea8fe5c2064 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0fe6265fb85e4499b4ea009486258251'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:41:35.155444 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-241740ac-223f-4e99-a822-5ea8fe5c2064 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0fe6265fb85e4499b4ea009486258251'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:41:35.156219 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-241740ac-223f-4e99-a822-5ea8fe5c2064 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0fe6265fb85e4499b4ea009486258251'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:41:35.185623 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1046/2075] 10.222.0.57 () {58 vars in 1223 bytes} [Fri Jul 1 16:41:34 2022] DELETE /identity/v3/users/0fe6265fb85e4499b4ea009486258251 => generated 0 bytes in 323 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 16:41:35.708787 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb396fa-c523-44ce-af63-9fa1256cec6e admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:41:35.709290 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-fbb396fa-c523-44ce-af63-9fa1256cec6e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:41:35.734145 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb396fa-c523-44ce-af63-9fa1256cec6e admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:41:35.932827 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a48f53e-3852-42b2-8a15-d9befc0880a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:41:35.934436 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a48f53e-3852-42b2-8a15-d9befc0880a7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:41:35.934831 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a48f53e-3852-42b2-8a15-d9befc0880a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:41:35.935374 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a48f53e-3852-42b2-8a15-d9befc0880a7 admin admin] PATH_INFO: `/v3/projects/e07de00e0b3a4983834b33d9a40ca306` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:41:35.941014 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a48f53e-3852-42b2-8a15-d9befc0880a7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e07de00e0b3a4983834b33d9a40ca306)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:41:35.943627 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a48f53e-3852-42b2-8a15-d9befc0880a7 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:41:36.026889 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:41:36.040776 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-2a48f53e-3852-42b2-8a15-d9befc0880a7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e07de00e0b3a4983834b33d9a40ca306', 'request_id': 'req-2a48f53e-3852-42b2-8a15-d9befc0880a7'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:41:36.063240 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-2a48f53e-3852-42b2-8a15-d9befc0880a7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e07de00e0b3a4983834b33d9a40ca306', 'request_id': 'req-2a48f53e-3852-42b2-8a15-d9befc0880a7'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:41:36.073125 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-2a48f53e-3852-42b2-8a15-d9befc0880a7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e07de00e0b3a4983834b33d9a40ca306', 'request_id': 'req-2a48f53e-3852-42b2-8a15-d9befc0880a7'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:41:36.093694 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-2a48f53e-3852-42b2-8a15-d9befc0880a7 admin admin] The token cache is being invalidate because project e07de00e0b3a4983834b33d9a40ca306 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80311) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 01 16:41:36.094180 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-2a48f53e-3852-42b2-8a15-d9befc0880a7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:41:36.095213 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-2a48f53e-3852-42b2-8a15-d9befc0880a7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:41:36.103959 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1030/2076] 10.222.0.57 () {58 vars in 1232 bytes} [Fri Jul 1 16:41:35 2022] DELETE /identity/v3/projects/e07de00e0b3a4983834b33d9a40ca306 => generated 0 bytes in 397 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 16:41:36.109049 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fa2d4e-bd52-40f2-8b4c-c38479cfc121 None None] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:41:36.109571 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fa2d4e-bd52-40f2-8b4c-c38479cfc121 None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:41:36.109958 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fa2d4e-bd52-40f2-8b4c-c38479cfc121 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:41:36.138194 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.auth.core [None req-69fa2d4e-bd52-40f2-8b4c-c38479cfc121 None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80312) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:41:36.139463 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-69fa2d4e-bd52-40f2-8b4c-c38479cfc121 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:41:36.183949 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:41:36.225538 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1047/2077] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 1 16:41:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 01 16:41:36.233766 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede62692-1beb-4abf-8564-61f9acd16d1e None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:41:36.234080 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede62692-1beb-4abf-8564-61f9acd16d1e None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:41:36.234375 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede62692-1beb-4abf-8564-61f9acd16d1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:41:36.263279 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-ede62692-1beb-4abf-8564-61f9acd16d1e None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:41:36.264534 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-ede62692-1beb-4abf-8564-61f9acd16d1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:41:36.287286 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1031/2078] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 1 16:41:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 16:41:36.288982 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69fa2d4e-bd52-40f2-8b4c-c38479cfc121 None None] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:41:36.301625 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69fa2d4e-bd52-40f2-8b4c-c38479cfc121 None None] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:41:36.308890 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48cf6dd-f35c-4b04-9977-d235cdd2424a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:41:36.309921 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48cf6dd-f35c-4b04-9977-d235cdd2424a admin admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:41:36.310303 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48cf6dd-f35c-4b04-9977-d235cdd2424a admin admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:41:36.310720 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48cf6dd-f35c-4b04-9977-d235cdd2424a admin admin] PATH_INFO: `/v3/domains` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:41:36.311562 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e48cf6dd-f35c-4b04-9977-d235cdd2424a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80312) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:41:36.311995 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e48cf6dd-f35c-4b04-9977-d235cdd2424a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:41:36.315268 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e48cf6dd-f35c-4b04-9977-d235cdd2424a admin admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:41:36.337179 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1048/2079] 10.222.0.57 () {58 vars in 1152 bytes} [Fri Jul 1 16:41:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:41:36.348093 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede62692-1beb-4abf-8564-61f9acd16d1e None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:41:36.360186 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede62692-1beb-4abf-8564-61f9acd16d1e None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:41:36.368270 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333434f1-02f3-46a0-a32d-c219d7b8d22d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:41:36.369254 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333434f1-02f3-46a0-a32d-c219d7b8d22d admin admin] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:41:36.369676 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333434f1-02f3-46a0-a32d-c219d7b8d22d admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:41:36.370136 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333434f1-02f3-46a0-a32d-c219d7b8d22d admin admin] PATH_INFO: `/v3/projects` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:41:36.371795 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333434f1-02f3-46a0-a32d-c219d7b8d22d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:41:36.373974 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333434f1-02f3-46a0-a32d-c219d7b8d22d admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:41:36.414704 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1032/2080] 10.222.0.57 () {60 vars in 1152 bytes} [Fri Jul 1 16:41:36 2022] POST /identity/v3/projects => generated 365 bytes in 67 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 16:41:36.418931 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48cf6dd-f35c-4b04-9977-d235cdd2424a admin admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:41:36.433734 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48cf6dd-f35c-4b04-9977-d235cdd2424a admin admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:41:36.593488 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e35c9e-93d7-4612-969d-4194a7c86aa7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:41:36.594293 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e35c9e-93d7-4612-969d-4194a7c86aa7 admin admin] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:41:36.595460 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e35c9e-93d7-4612-969d-4194a7c86aa7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:41:36.595923 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e35c9e-93d7-4612-969d-4194a7c86aa7 admin admin] PATH_INFO: `/v3/users` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:41:36.597312 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e35c9e-93d7-4612-969d-4194a7c86aa7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:41:36.600960 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e35c9e-93d7-4612-969d-4194a7c86aa7 admin admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:41:36.666901 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1049/2081] 10.222.0.57 () {60 vars in 1143 bytes} [Fri Jul 1 16:41:36 2022] POST /identity/v3/users => generated 283 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 01 16:41:36.670544 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333434f1-02f3-46a0-a32d-c219d7b8d22d admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:41:36.693075 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333434f1-02f3-46a0-a32d-c219d7b8d22d admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:41:36.711634 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8e6bff-8c36-4b6e-8053-cd987af425c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:41:36.712361 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8e6bff-8c36-4b6e-8053-cd987af425c0 admin admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:41:36.713057 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8e6bff-8c36-4b6e-8053-cd987af425c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:41:36.713534 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8e6bff-8c36-4b6e-8053-cd987af425c0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:41:36.714509 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8e6bff-8c36-4b6e-8053-cd987af425c0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:41:36.717025 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8e6bff-8c36-4b6e-8053-cd987af425c0 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:41:36.761644 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1033/2082] 10.222.0.57 () {58 vars in 1121 bytes} [Fri Jul 1 16:41:36 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 16:41:36.766060 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e35c9e-93d7-4612-969d-4194a7c86aa7 admin admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:41:36.784100 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e35c9e-93d7-4612-969d-4194a7c86aa7 admin admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:41:36.797287 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbbc084e-a4cf-4eac-803b-ae1e2ffda94b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:41:36.798771 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbc084e-a4cf-4eac-803b-ae1e2ffda94b admin admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:41:36.799088 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbc084e-a4cf-4eac-803b-ae1e2ffda94b admin admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:41:36.799457 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbc084e-a4cf-4eac-803b-ae1e2ffda94b admin admin] PATH_INFO: `/v3/roles` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:41:36.800127 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbbc084e-a4cf-4eac-803b-ae1e2ffda94b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:41:36.801774 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbbc084e-a4cf-4eac-803b-ae1e2ffda94b admin admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:41:36.827215 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1050/2083] 10.222.0.57 () {58 vars in 1121 bytes} [Fri Jul 1 16:41:36 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 16:41:36.832201 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8e6bff-8c36-4b6e-8053-cd987af425c0 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:41:36.844497 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8e6bff-8c36-4b6e-8053-cd987af425c0 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:41:36.855739 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ecab008-22f1-415b-b1ed-c0d8740868cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:41:36.856651 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecab008-22f1-415b-b1ed-c0d8740868cc admin admin] REQUEST_METHOD: `PUT` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:41:36.856932 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecab008-22f1-415b-b1ed-c0d8740868cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:41:36.857209 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecab008-22f1-415b-b1ed-c0d8740868cc admin admin] PATH_INFO: `/v3/projects/9d9fd8d9025748b6bfae532255c3e47d/users/502e03934fcf4bdd8bf416f18b22684c/roles/983be40bdab04c478eaa90c6b6a46b14` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:41:36.877397 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ecab008-22f1-415b-b1ed-c0d8740868cc admin admin] RBAC: Authorizing `identity:create_grant(project_id=9d9fd8d9025748b6bfae532255c3e47d, user_id=502e03934fcf4bdd8bf416f18b22684c, role_id=983be40bdab04c478eaa90c6b6a46b14)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:41:36.880372 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ecab008-22f1-415b-b1ed-c0d8740868cc admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:41:36.905279 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1034/2084] 10.222.0.57 () {60 vars in 1482 bytes} [Fri Jul 1 16:41:36 2022] PUT /identity/v3/projects/9d9fd8d9025748b6bfae532255c3e47d/users/502e03934fcf4bdd8bf416f18b22684c/roles/983be40bdab04c478eaa90c6b6a46b14 => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 16:41:36.917269 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d065f9-b3d7-4179-9ca6-6f63b4930580 None None] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:41:36.917600 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d065f9-b3d7-4179-9ca6-6f63b4930580 None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:41:36.917844 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d065f9-b3d7-4179-9ca6-6f63b4930580 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:41:36.953213 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.auth.core [None req-18d065f9-b3d7-4179-9ca6-6f63b4930580 None None] MFA Rules not processed for user `502e03934fcf4bdd8bf416f18b22684c`. Rule list: `[]` (Enabled: `True`). {{(pid=80312) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:41:36.957244 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-18d065f9-b3d7-4179-9ca6-6f63b4930580 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:41:37.159639 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1051/2085] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 1 16:41:36 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 243 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 16:41:37.163976 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ecab008-22f1-415b-b1ed-c0d8740868cc admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:41:37.287093 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ecab008-22f1-415b-b1ed-c0d8740868cc admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:41:37.293058 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08150904-8a38-4c2c-bde7-1f47228c9826 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:41:37.293840 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08150904-8a38-4c2c-bde7-1f47228c9826 admin admin] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:41:37.294261 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08150904-8a38-4c2c-bde7-1f47228c9826 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:41:37.294579 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08150904-8a38-4c2c-bde7-1f47228c9826 admin admin] PATH_INFO: `/v3/projects` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:41:37.295325 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08150904-8a38-4c2c-bde7-1f47228c9826 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:41:37.299856 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08150904-8a38-4c2c-bde7-1f47228c9826 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:41:37.358093 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d065f9-b3d7-4179-9ca6-6f63b4930580 None None] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:41:37.365849 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1035/2086] 10.222.0.57 () {60 vars in 1152 bytes} [Fri Jul 1 16:41:37 2022] POST /identity/v3/projects => generated 367 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 16:41:37.373539 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d065f9-b3d7-4179-9ca6-6f63b4930580 None None] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:41:37.411666 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:41:37.510971 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922f6323-f782-4edc-b159-79dbb80f6fa2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:41:37.517148 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922f6323-f782-4edc-b159-79dbb80f6fa2 admin admin] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:41:37.517592 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922f6323-f782-4edc-b159-79dbb80f6fa2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:41:37.517883 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922f6323-f782-4edc-b159-79dbb80f6fa2 admin admin] PATH_INFO: `/v3/users` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:41:37.518685 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-922f6323-f782-4edc-b159-79dbb80f6fa2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:41:37.520706 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-922f6323-f782-4edc-b159-79dbb80f6fa2 admin admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:41:37.617245 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08150904-8a38-4c2c-bde7-1f47228c9826 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:41:37.628940 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08150904-8a38-4c2c-bde7-1f47228c9826 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:41:37.636796 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1052/2087] 10.222.0.57 () {60 vars in 1143 bytes} [Fri Jul 1 16:41:37 2022] POST /identity/v3/users => generated 290 bytes in 274 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 16:41:37.655786 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9371d676-1ce3-46b0-bd12-6f78207901a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:41:37.656473 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9371d676-1ce3-46b0-bd12-6f78207901a5 admin admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:41:37.656760 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9371d676-1ce3-46b0-bd12-6f78207901a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:41:37.657033 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9371d676-1ce3-46b0-bd12-6f78207901a5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:41:37.658061 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9371d676-1ce3-46b0-bd12-6f78207901a5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:41:37.659754 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9371d676-1ce3-46b0-bd12-6f78207901a5 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:41:37.689361 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1036/2088] 10.222.0.57 () {58 vars in 1121 bytes} [Fri Jul 1 16:41:37 2022] GET /identity/v3/roles => generated 1420 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 01 16:41:37.693444 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922f6323-f782-4edc-b159-79dbb80f6fa2 admin admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:41:37.723637 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922f6323-f782-4edc-b159-79dbb80f6fa2 admin admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:41:37.742667 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b4ded0-1fb3-4ff5-a998-5caf8a580c8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:41:37.744050 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b4ded0-1fb3-4ff5-a998-5caf8a580c8f admin admin] REQUEST_METHOD: `PUT` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:41:37.744050 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b4ded0-1fb3-4ff5-a998-5caf8a580c8f admin admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:41:37.744251 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b4ded0-1fb3-4ff5-a998-5caf8a580c8f admin admin] PATH_INFO: `/v3/projects/e531614bcdf14dda87291017e61687ad/users/edf0a8606c434e11a71650e5eab80352/roles/84ee739146934c1ab32c501ff8093644` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:41:37.778273 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b4ded0-1fb3-4ff5-a998-5caf8a580c8f admin admin] RBAC: Authorizing `identity:create_grant(project_id=e531614bcdf14dda87291017e61687ad, user_id=edf0a8606c434e11a71650e5eab80352, role_id=84ee739146934c1ab32c501ff8093644)` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:41:37.781002 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b4ded0-1fb3-4ff5-a998-5caf8a580c8f admin admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:41:37.808026 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1053/2089] 10.222.0.57 () {60 vars in 1482 bytes} [Fri Jul 1 16:41:37 2022] PUT /identity/v3/projects/e531614bcdf14dda87291017e61687ad/users/edf0a8606c434e11a71650e5eab80352/roles/84ee739146934c1ab32c501ff8093644 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 16:41:37.812254 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9371d676-1ce3-46b0-bd12-6f78207901a5 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:41:37.824631 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9371d676-1ce3-46b0-bd12-6f78207901a5 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:41:37.921603 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:41:37.998999 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dac794a-af77-48c9-87eb-b4279beaa2dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:41:38.000841 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dac794a-af77-48c9-87eb-b4279beaa2dc admin admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:41:38.001009 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dac794a-af77-48c9-87eb-b4279beaa2dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:41:38.005029 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dac794a-af77-48c9-87eb-b4279beaa2dc admin admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:41:38.008052 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dac794a-af77-48c9-87eb-b4279beaa2dc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:41:38.016904 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dac794a-af77-48c9-87eb-b4279beaa2dc admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:41:38.043443 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1037/2090] 10.222.0.57 () {58 vars in 1121 bytes} [Fri Jul 1 16:41:37 2022] GET /identity/v3/roles => generated 1420 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 16:41:38.044916 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b4ded0-1fb3-4ff5-a998-5caf8a580c8f admin admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:41:38.057969 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b4ded0-1fb3-4ff5-a998-5caf8a580c8f admin admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:41:38.074283 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1788a810-da57-48cc-b2c1-e5ddb3c737c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:41:38.075547 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1788a810-da57-48cc-b2c1-e5ddb3c737c8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:41:38.076008 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1788a810-da57-48cc-b2c1-e5ddb3c737c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:41:38.076703 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1788a810-da57-48cc-b2c1-e5ddb3c737c8 admin admin] PATH_INFO: `/v3/projects/e531614bcdf14dda87291017e61687ad/users/edf0a8606c434e11a71650e5eab80352/roles/84ee739146934c1ab32c501ff8093644` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:41:38.081271 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1788a810-da57-48cc-b2c1-e5ddb3c737c8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e531614bcdf14dda87291017e61687ad, user_id=edf0a8606c434e11a71650e5eab80352, role_id=84ee739146934c1ab32c501ff8093644)` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:41:38.090292 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1788a810-da57-48cc-b2c1-e5ddb3c737c8 admin admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:41:38.132423 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661c2e20-7719-4802-9d07-a887a09cb554 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:41:38.133004 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661c2e20-7719-4802-9d07-a887a09cb554 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:41:38.133489 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661c2e20-7719-4802-9d07-a887a09cb554 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:41:38.137879 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1054/2091] 10.222.0.57 () {60 vars in 1482 bytes} [Fri Jul 1 16:41:38 2022] PUT /identity/v3/projects/e531614bcdf14dda87291017e61687ad/users/edf0a8606c434e11a71650e5eab80352/roles/84ee739146934c1ab32c501ff8093644 => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 16:41:38.167321 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-661c2e20-7719-4802-9d07-a887a09cb554 None None] MFA Rules not processed for user `edf0a8606c434e11a71650e5eab80352`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:41:38.168345 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-661c2e20-7719-4802-9d07-a887a09cb554 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:41:38.256730 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1038/2092] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 1 16:41:38 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 16:41:38.264677 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1788a810-da57-48cc-b2c1-e5ddb3c737c8 admin admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:41:38.267773 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-1788a810-da57-48cc-b2c1-e5ddb3c737c8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:41:38.379469 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1788a810-da57-48cc-b2c1-e5ddb3c737c8 admin admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:41:38.384999 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c92f97-db40-4675-947c-2b9ac1580c59 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbddf971a2354bbdbb9da1d63f3abf88', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0788e6725a84e57a4174284c1845755', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:41:38.386478 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c92f97-db40-4675-947c-2b9ac1580c59 service cinder] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:41:38.386778 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c92f97-db40-4675-947c-2b9ac1580c59 service cinder] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:41:38.387069 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c92f97-db40-4675-947c-2b9ac1580c59 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:41:38.402299 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c92f97-db40-4675-947c-2b9ac1580c59 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:41:38.404853 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c92f97-db40-4675-947c-2b9ac1580c59 service cinder] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:41:38.435445 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1055/2093] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 1 16:41:38 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 16:41:38.544041 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661c2e20-7719-4802-9d07-a887a09cb554 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:41:38.560697 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661c2e20-7719-4802-9d07-a887a09cb554 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:41:38.566232 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8c058c-26c8-4782-8e40-536713e88e65 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbddf971a2354bbdbb9da1d63f3abf88', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0788e6725a84e57a4174284c1845755', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:41:38.566970 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8c058c-26c8-4782-8e40-536713e88e65 service cinder] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:41:38.567260 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8c058c-26c8-4782-8e40-536713e88e65 service cinder] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:41:38.567624 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8c058c-26c8-4782-8e40-536713e88e65 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:41:38.685945 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f8c058c-26c8-4782-8e40-536713e88e65 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:41:38.687754 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f8c058c-26c8-4782-8e40-536713e88e65 service cinder] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:41:38.700390 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1039/2094] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 1 16:41:38 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 16:41:39.875811 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2b356d-b036-4feb-9ed2-a17dcd31e8ce None None] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:41:39.876985 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2b356d-b036-4feb-9ed2-a17dcd31e8ce None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:41:39.876985 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2b356d-b036-4feb-9ed2-a17dcd31e8ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:41:39.914078 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.auth.core [None req-5c2b356d-b036-4feb-9ed2-a17dcd31e8ce None None] MFA Rules not processed for user `4a269ade40e44ad986f32ce794371e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=80312) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:41:39.915391 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-5c2b356d-b036-4feb-9ed2-a17dcd31e8ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:41:39.999275 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1056/2095] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 1 16:41:39 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 16:41:40.016036 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8c058c-26c8-4782-8e40-536713e88e65 service cinder] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:41:40.018680 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-6f8c058c-26c8-4782-8e40-536713e88e65 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:41:40.100186 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8c058c-26c8-4782-8e40-536713e88e65 service cinder] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:41:40.112987 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f84d8ae-3c82-4623-b1c6-70f446fec4e5 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd028715bfea43099e7c6cde429ecf92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0788e6725a84e57a4174284c1845755', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:41:40.112987 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f84d8ae-3c82-4623-b1c6-70f446fec4e5 service glance] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:41:40.112987 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f84d8ae-3c82-4623-b1c6-70f446fec4e5 service glance] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:41:40.112987 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f84d8ae-3c82-4623-b1c6-70f446fec4e5 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:41:40.113391 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-9f84d8ae-3c82-4623-b1c6-70f446fec4e5 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:41:40.120845 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:41:40.173030 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f84d8ae-3c82-4623-b1c6-70f446fec4e5 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:41:40.174782 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f84d8ae-3c82-4623-b1c6-70f446fec4e5 service glance] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:41:40.184358 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1040/2096] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 1 16:41:40 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 01 16:41:40.337815 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d31386-e9be-4348-8d79-0cb583406537 None None] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:41:40.338308 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d31386-e9be-4348-8d79-0cb583406537 None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:41:40.338644 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d31386-e9be-4348-8d79-0cb583406537 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:41:40.363428 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.auth.core [None req-00d31386-e9be-4348-8d79-0cb583406537 None None] MFA Rules not processed for user `c4de1b1eeaed4fc5ae7200b2c7a90957`. Rule list: `[]` (Enabled: `True`). {{(pid=80312) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:41:40.364815 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-00d31386-e9be-4348-8d79-0cb583406537 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:41:40.372478 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:41:40.427291 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1057/2097] 10.222.0.57 () {58 vars in 1009 bytes} [Fri Jul 1 16:41:40 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 16:41:40.469126 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f84d8ae-3c82-4623-b1c6-70f446fec4e5 service glance] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:41:40.471116 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-9f84d8ae-3c82-4623-b1c6-70f446fec4e5 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:41:40.539828 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f84d8ae-3c82-4623-b1c6-70f446fec4e5 service glance] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:41:40.545011 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6155ed7-d009-4e8e-829b-bef34767a766 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '859fe410c2d2450fb803c6244839f27c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0788e6725a84e57a4174284c1845755', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:41:40.545662 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6155ed7-d009-4e8e-829b-bef34767a766 service swift] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:41:40.545946 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6155ed7-d009-4e8e-829b-bef34767a766 service swift] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:41:40.546227 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6155ed7-d009-4e8e-829b-bef34767a766 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:41:40.555810 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6155ed7-d009-4e8e-829b-bef34767a766 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:41:40.557518 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6155ed7-d009-4e8e-829b-bef34767a766 service swift] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:41:40.566172 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1041/2098] 10.222.0.57 () {60 vars in 1384 bytes} [Fri Jul 1 16:41:40 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 01 16:41:46.847632 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d31386-e9be-4348-8d79-0cb583406537 None None] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:41:46.883324 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6155ed7-d009-4e8e-829b-bef34767a766 service swift] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:41:46.885457 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-c6155ed7-d009-4e8e-829b-bef34767a766 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:41:46.952349 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d31386-e9be-4348-8d79-0cb583406537 None None] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:41:46.960843 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4541558-6364-4098-b665-f101a5fa9d00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:41:46.963168 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4541558-6364-4098-b665-f101a5fa9d00 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:41:46.963847 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4541558-6364-4098-b665-f101a5fa9d00 admin admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:41:46.964343 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4541558-6364-4098-b665-f101a5fa9d00 admin admin] PATH_INFO: `/v3/users/502e03934fcf4bdd8bf416f18b22684c` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:41:46.966604 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4541558-6364-4098-b665-f101a5fa9d00 admin admin] RBAC: Authorizing `identity:delete_user(user_id=502e03934fcf4bdd8bf416f18b22684c)` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:41:46.969757 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4541558-6364-4098-b665-f101a5fa9d00 admin admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:41:47.084743 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6155ed7-d009-4e8e-829b-bef34767a766 service swift] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:41:47.086449 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-b4541558-6364-4098-b665-f101a5fa9d00 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '502e03934fcf4bdd8bf416f18b22684c'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:41:47.096408 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a231491-d6cc-4192-9488-2091084f60b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:41:47.098022 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a231491-d6cc-4192-9488-2091084f60b8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:41:47.098319 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a231491-d6cc-4192-9488-2091084f60b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:41:47.098622 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a231491-d6cc-4192-9488-2091084f60b8 admin admin] PATH_INFO: `/v3/users/25f8caac070f4d3daa1777fe145432a4` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:41:47.100353 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a231491-d6cc-4192-9488-2091084f60b8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=25f8caac070f4d3daa1777fe145432a4)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:41:47.102594 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a231491-d6cc-4192-9488-2091084f60b8 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:41:47.120056 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-b4541558-6364-4098-b665-f101a5fa9d00 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '502e03934fcf4bdd8bf416f18b22684c'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:41:47.139083 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-b4541558-6364-4098-b665-f101a5fa9d00 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '502e03934fcf4bdd8bf416f18b22684c'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:41:47.141877 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-b4541558-6364-4098-b665-f101a5fa9d00 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '502e03934fcf4bdd8bf416f18b22684c'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:41:47.143296 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-b4541558-6364-4098-b665-f101a5fa9d00 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '502e03934fcf4bdd8bf416f18b22684c'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:41:47.171144 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1058/2099] 10.222.0.57 () {58 vars in 1223 bytes} [Fri Jul 1 16:41:46 2022] DELETE /identity/v3/users/502e03934fcf4bdd8bf416f18b22684c => generated 0 bytes in 324 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 16:41:47.175724 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483f77fc-9189-4c62-a669-d94bffd0734a None None] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:41:47.176141 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483f77fc-9189-4c62-a669-d94bffd0734a None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:41:47.176523 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483f77fc-9189-4c62-a669-d94bffd0734a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:41:47.230136 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.auth.core [None req-483f77fc-9189-4c62-a669-d94bffd0734a None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80312) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:41:47.231545 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-483f77fc-9189-4c62-a669-d94bffd0734a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:41:47.233997 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:41:47.236692 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-8a231491-d6cc-4192-9488-2091084f60b8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '25f8caac070f4d3daa1777fe145432a4'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:41:47.239694 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-8a231491-d6cc-4192-9488-2091084f60b8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '25f8caac070f4d3daa1777fe145432a4'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:41:47.240974 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-8a231491-d6cc-4192-9488-2091084f60b8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '25f8caac070f4d3daa1777fe145432a4'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:41:47.251714 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-8a231491-d6cc-4192-9488-2091084f60b8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '25f8caac070f4d3daa1777fe145432a4'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:41:47.273653 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-8a231491-d6cc-4192-9488-2091084f60b8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '25f8caac070f4d3daa1777fe145432a4'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:41:47.289621 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1042/2100] 10.222.0.57 () {58 vars in 1223 bytes} [Fri Jul 1 16:41:46 2022] DELETE /identity/v3/users/25f8caac070f4d3daa1777fe145432a4 => generated 0 bytes in 407 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 01 16:41:47.295076 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0024ea-3193-4bc3-896d-08101859e1ad None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:41:47.295726 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0024ea-3193-4bc3-896d-08101859e1ad None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:41:47.295726 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0024ea-3193-4bc3-896d-08101859e1ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:41:47.312562 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:41:47.344453 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-7e0024ea-3193-4bc3-896d-08101859e1ad None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:41:47.345957 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-7e0024ea-3193-4bc3-896d-08101859e1ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:41:47.379932 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1059/2101] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 1 16:41:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 16:41:47.429074 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483f77fc-9189-4c62-a669-d94bffd0734a None None] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:41:47.430681 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-483f77fc-9189-4c62-a669-d94bffd0734a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:41:47.487659 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1043/2102] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 1 16:41:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 01 16:41:47.533159 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0024ea-3193-4bc3-896d-08101859e1ad None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:41:47.534724 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-7e0024ea-3193-4bc3-896d-08101859e1ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:41:47.561231 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483f77fc-9189-4c62-a669-d94bffd0734a None None] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:41:47.644821 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0024ea-3193-4bc3-896d-08101859e1ad None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:41:47.651686 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79aa2e6d-e12a-4143-b4c1-a3b087b04980 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ed200ce29684c5f80354fa9a8638fe2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0788e6725a84e57a4174284c1845755', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:41:47.652382 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79aa2e6d-e12a-4143-b4c1-a3b087b04980 service neutron] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:41:47.652659 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79aa2e6d-e12a-4143-b4c1-a3b087b04980 service neutron] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:41:47.653081 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79aa2e6d-e12a-4143-b4c1-a3b087b04980 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:41:47.660921 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b132887-7001-42b4-a2b0-adf83f36e0cd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ed200ce29684c5f80354fa9a8638fe2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0788e6725a84e57a4174284c1845755', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:41:47.661758 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b132887-7001-42b4-a2b0-adf83f36e0cd service neutron] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:41:47.662163 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b132887-7001-42b4-a2b0-adf83f36e0cd service neutron] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:41:47.662572 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b132887-7001-42b4-a2b0-adf83f36e0cd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:41:47.669488 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79aa2e6d-e12a-4143-b4c1-a3b087b04980 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:41:47.671457 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79aa2e6d-e12a-4143-b4c1-a3b087b04980 service neutron] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:41:47.675468 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b132887-7001-42b4-a2b0-adf83f36e0cd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:41:47.677711 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b132887-7001-42b4-a2b0-adf83f36e0cd service neutron] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:41:47.684308 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1044/2103] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 1 16:41:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 01 16:41:47.690533 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1060/2104] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 1 16:41:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 262 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 01 16:41:48.160920 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79aa2e6d-e12a-4143-b4c1-a3b087b04980 service neutron] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:41:48.163770 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-79aa2e6d-e12a-4143-b4c1-a3b087b04980 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:41:48.202930 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79aa2e6d-e12a-4143-b4c1-a3b087b04980 service neutron] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:41:48.235093 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aed89da-ab27-4c59-b135-c915f463934f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:41:48.239147 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aed89da-ab27-4c59-b135-c915f463934f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:41:48.240424 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aed89da-ab27-4c59-b135-c915f463934f admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:41:48.242869 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aed89da-ab27-4c59-b135-c915f463934f admin admin] PATH_INFO: `/v3/projects/6ee973569cba400f813060ab544d4eab` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:41:48.246588 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aed89da-ab27-4c59-b135-c915f463934f admin admin] RBAC: Authorizing `identity:delete_project(project_id=6ee973569cba400f813060ab544d4eab)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:41:48.254317 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aed89da-ab27-4c59-b135-c915f463934f admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:41:48.352481 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b132887-7001-42b4-a2b0-adf83f36e0cd service neutron] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:41:48.355421 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-7b132887-7001-42b4-a2b0-adf83f36e0cd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:41:48.386096 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-4aed89da-ab27-4c59-b135-c915f463934f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6ee973569cba400f813060ab544d4eab', 'request_id': 'req-4aed89da-ab27-4c59-b135-c915f463934f'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:41:48.422116 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b132887-7001-42b4-a2b0-adf83f36e0cd service neutron] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:41:48.423382 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-4aed89da-ab27-4c59-b135-c915f463934f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6ee973569cba400f813060ab544d4eab', 'request_id': 'req-4aed89da-ab27-4c59-b135-c915f463934f'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:41:48.427852 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=90,keystone:INSERT=1,keystone:DELETE=9 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:41:48.446110 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-4aed89da-ab27-4c59-b135-c915f463934f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6ee973569cba400f813060ab544d4eab', 'request_id': 'req-4aed89da-ab27-4c59-b135-c915f463934f'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:41:48.484226 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-4aed89da-ab27-4c59-b135-c915f463934f admin admin] The token cache is being invalidate because project 6ee973569cba400f813060ab544d4eab was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80311) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 01 16:41:48.486784 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-4aed89da-ab27-4c59-b135-c915f463934f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:41:48.487667 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-4aed89da-ab27-4c59-b135-c915f463934f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:41:48.497689 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1045/2105] 10.222.0.57 () {58 vars in 1232 bytes} [Fri Jul 1 16:41:48 2022] DELETE /identity/v3/projects/6ee973569cba400f813060ab544d4eab => generated 0 bytes in 334 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 16:41:48.501965 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aed89da-ab27-4c59-b135-c915f463934f admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:41:48.506781 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-4aed89da-ab27-4c59-b135-c915f463934f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:41:48.542039 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4f38cf-2aa8-4f11-abd8-dad135a8036c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:41:48.552372 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4f38cf-2aa8-4f11-abd8-dad135a8036c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:41:48.552621 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4f38cf-2aa8-4f11-abd8-dad135a8036c admin admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:41:48.554044 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4f38cf-2aa8-4f11-abd8-dad135a8036c admin admin] PATH_INFO: `/v3/projects/9d9fd8d9025748b6bfae532255c3e47d` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:41:48.574656 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c4f38cf-2aa8-4f11-abd8-dad135a8036c admin admin] RBAC: Authorizing `identity:delete_project(project_id=9d9fd8d9025748b6bfae532255c3e47d)` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:41:48.577411 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c4f38cf-2aa8-4f11-abd8-dad135a8036c admin admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:41:48.679002 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:41:48.702586 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aed89da-ab27-4c59-b135-c915f463934f admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:41:48.704300 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-1c4f38cf-2aa8-4f11-abd8-dad135a8036c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9d9fd8d9025748b6bfae532255c3e47d', 'request_id': 'req-1c4f38cf-2aa8-4f11-abd8-dad135a8036c'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:41:48.721116 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-1c4f38cf-2aa8-4f11-abd8-dad135a8036c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9d9fd8d9025748b6bfae532255c3e47d', 'request_id': 'req-1c4f38cf-2aa8-4f11-abd8-dad135a8036c'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:41:48.731699 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-1c4f38cf-2aa8-4f11-abd8-dad135a8036c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9d9fd8d9025748b6bfae532255c3e47d', 'request_id': 'req-1c4f38cf-2aa8-4f11-abd8-dad135a8036c'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:41:48.770076 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-1c4f38cf-2aa8-4f11-abd8-dad135a8036c admin admin] The token cache is being invalidate because project 9d9fd8d9025748b6bfae532255c3e47d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80312) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 01 16:41:48.770650 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-1c4f38cf-2aa8-4f11-abd8-dad135a8036c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:41:48.786287 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-1c4f38cf-2aa8-4f11-abd8-dad135a8036c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:41:48.806019 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1061/2106] 10.222.0.57 () {58 vars in 1232 bytes} [Fri Jul 1 16:41:48 2022] DELETE /identity/v3/projects/9d9fd8d9025748b6bfae532255c3e47d => generated 0 bytes in 451 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 16:41:48.807559 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4f38cf-2aa8-4f11-abd8-dad135a8036c admin admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:41:48.812340 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-1c4f38cf-2aa8-4f11-abd8-dad135a8036c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:41:48.901092 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeaa159c-7f60-47c9-a4b3-7d33a6d9f8e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:41:48.901849 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeaa159c-7f60-47c9-a4b3-7d33a6d9f8e8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:41:48.902241 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeaa159c-7f60-47c9-a4b3-7d33a6d9f8e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:41:48.902626 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeaa159c-7f60-47c9-a4b3-7d33a6d9f8e8 admin admin] PATH_INFO: `/v3/users/21d2f2ba617a41bd9d4640f8a7f5488c` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:41:48.904112 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeaa159c-7f60-47c9-a4b3-7d33a6d9f8e8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=21d2f2ba617a41bd9d4640f8a7f5488c)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:41:48.907742 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeaa159c-7f60-47c9-a4b3-7d33a6d9f8e8 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:41:49.101959 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4f38cf-2aa8-4f11-abd8-dad135a8036c admin admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:41:49.103987 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-eeaa159c-7f60-47c9-a4b3-7d33a6d9f8e8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '21d2f2ba617a41bd9d4640f8a7f5488c'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:41:49.106703 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-eeaa159c-7f60-47c9-a4b3-7d33a6d9f8e8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '21d2f2ba617a41bd9d4640f8a7f5488c'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:41:49.107372 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-eeaa159c-7f60-47c9-a4b3-7d33a6d9f8e8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '21d2f2ba617a41bd9d4640f8a7f5488c'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:41:49.107485 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd99418-451b-4f2a-ba5a-d0fc600fa682 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:41:49.109119 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd99418-451b-4f2a-ba5a-d0fc600fa682 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:41:49.110294 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd99418-451b-4f2a-ba5a-d0fc600fa682 admin admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:41:49.110645 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd99418-451b-4f2a-ba5a-d0fc600fa682 admin admin] PATH_INFO: `/v3/users/edf0a8606c434e11a71650e5eab80352` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:41:49.112173 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd99418-451b-4f2a-ba5a-d0fc600fa682 admin admin] RBAC: Authorizing `identity:delete_user(user_id=edf0a8606c434e11a71650e5eab80352)` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:41:49.114935 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd99418-451b-4f2a-ba5a-d0fc600fa682 admin admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:41:49.120306 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-eeaa159c-7f60-47c9-a4b3-7d33a6d9f8e8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '21d2f2ba617a41bd9d4640f8a7f5488c'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:41:49.149483 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-eeaa159c-7f60-47c9-a4b3-7d33a6d9f8e8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '21d2f2ba617a41bd9d4640f8a7f5488c'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:41:49.178180 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1046/2107] 10.222.0.57 () {58 vars in 1223 bytes} [Fri Jul 1 16:41:48 2022] DELETE /identity/v3/users/21d2f2ba617a41bd9d4640f8a7f5488c => generated 0 bytes in 679 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 16:41:49.194269 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-edd99418-451b-4f2a-ba5a-d0fc600fa682 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'edf0a8606c434e11a71650e5eab80352'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:41:49.211898 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-edd99418-451b-4f2a-ba5a-d0fc600fa682 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'edf0a8606c434e11a71650e5eab80352'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:41:49.225769 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-edd99418-451b-4f2a-ba5a-d0fc600fa682 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'edf0a8606c434e11a71650e5eab80352'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:41:49.227169 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-edd99418-451b-4f2a-ba5a-d0fc600fa682 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'edf0a8606c434e11a71650e5eab80352'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:41:49.228301 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-edd99418-451b-4f2a-ba5a-d0fc600fa682 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'edf0a8606c434e11a71650e5eab80352'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:41:49.260063 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1062/2108] 10.222.0.57 () {58 vars in 1223 bytes} [Fri Jul 1 16:41:48 2022] DELETE /identity/v3/users/edf0a8606c434e11a71650e5eab80352 => generated 0 bytes in 453 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 16:41:49.826322 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeaa159c-7f60-47c9-a4b3-7d33a6d9f8e8 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:41:49.828167 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-eeaa159c-7f60-47c9-a4b3-7d33a6d9f8e8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:41:49.898984 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=87,keystone:INSERT=2,keystone:DELETE=11 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:41:49.923680 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd99418-451b-4f2a-ba5a-d0fc600fa682 admin admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:41:49.927991 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-edd99418-451b-4f2a-ba5a-d0fc600fa682 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:41:50.009552 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeaa159c-7f60-47c9-a4b3-7d33a6d9f8e8 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:41:50.031136 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:41:50.070533 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd99418-451b-4f2a-ba5a-d0fc600fa682 admin admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:41:50.078397 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904aadd8-05ad-47f9-b905-55a1822410b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:41:50.080863 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904aadd8-05ad-47f9-b905-55a1822410b0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:41:50.081191 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904aadd8-05ad-47f9-b905-55a1822410b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:41:50.081504 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904aadd8-05ad-47f9-b905-55a1822410b0 admin admin] PATH_INFO: `/v3/users/ea27a708e2c84ccc905ae783524bd12f` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:41:50.083415 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904aadd8-05ad-47f9-b905-55a1822410b0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ea27a708e2c84ccc905ae783524bd12f)` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:41:50.087798 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904aadd8-05ad-47f9-b905-55a1822410b0 admin admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:41:50.094954 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd230c4b-431b-4341-9207-0bfdf59a1de8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:41:50.095834 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd230c4b-431b-4341-9207-0bfdf59a1de8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:41:50.096126 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd230c4b-431b-4341-9207-0bfdf59a1de8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:41:50.097031 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd230c4b-431b-4341-9207-0bfdf59a1de8 admin admin] PATH_INFO: `/v3/projects/e531614bcdf14dda87291017e61687ad` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:41:50.098936 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd230c4b-431b-4341-9207-0bfdf59a1de8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e531614bcdf14dda87291017e61687ad)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:41:50.101621 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd230c4b-431b-4341-9207-0bfdf59a1de8 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:41:50.172028 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-cd230c4b-431b-4341-9207-0bfdf59a1de8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e531614bcdf14dda87291017e61687ad', 'request_id': 'req-cd230c4b-431b-4341-9207-0bfdf59a1de8'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:41:50.180553 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-904aadd8-05ad-47f9-b905-55a1822410b0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ea27a708e2c84ccc905ae783524bd12f'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:41:50.182494 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-cd230c4b-431b-4341-9207-0bfdf59a1de8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e531614bcdf14dda87291017e61687ad', 'request_id': 'req-cd230c4b-431b-4341-9207-0bfdf59a1de8'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:41:50.202112 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-cd230c4b-431b-4341-9207-0bfdf59a1de8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e531614bcdf14dda87291017e61687ad', 'request_id': 'req-cd230c4b-431b-4341-9207-0bfdf59a1de8'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:41:50.208136 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-904aadd8-05ad-47f9-b905-55a1822410b0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ea27a708e2c84ccc905ae783524bd12f'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:41:50.224081 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-cd230c4b-431b-4341-9207-0bfdf59a1de8 admin admin] The token cache is being invalidate because project e531614bcdf14dda87291017e61687ad was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80311) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 01 16:41:50.224931 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-cd230c4b-431b-4341-9207-0bfdf59a1de8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:41:50.225347 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-cd230c4b-431b-4341-9207-0bfdf59a1de8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:41:50.225659 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-904aadd8-05ad-47f9-b905-55a1822410b0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ea27a708e2c84ccc905ae783524bd12f'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:41:50.228139 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-904aadd8-05ad-47f9-b905-55a1822410b0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ea27a708e2c84ccc905ae783524bd12f'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:41:50.229671 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-904aadd8-05ad-47f9-b905-55a1822410b0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ea27a708e2c84ccc905ae783524bd12f'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:41:50.233673 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1047/2109] 10.222.0.57 () {58 vars in 1232 bytes} [Fri Jul 1 16:41:49 2022] DELETE /identity/v3/projects/e531614bcdf14dda87291017e61687ad => generated 0 bytes in 403 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 16:41:50.234725 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd230c4b-431b-4341-9207-0bfdf59a1de8 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:41:50.237098 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-cd230c4b-431b-4341-9207-0bfdf59a1de8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:41:50.270015 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1063/2110] 10.222.0.57 () {58 vars in 1223 bytes} [Fri Jul 1 16:41:49 2022] DELETE /identity/v3/users/ea27a708e2c84ccc905ae783524bd12f => generated 0 bytes in 347 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 16:41:50.271727 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d9221c-9d97-4577-93c0-7f4e046d43c3 None None] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:41:50.272184 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d9221c-9d97-4577-93c0-7f4e046d43c3 None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:41:50.273055 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d9221c-9d97-4577-93c0-7f4e046d43c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:41:50.297194 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.auth.core [None req-03d9221c-9d97-4577-93c0-7f4e046d43c3 None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80312) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:41:50.298784 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-03d9221c-9d97-4577-93c0-7f4e046d43c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:41:50.326781 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:41:50.354578 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd230c4b-431b-4341-9207-0bfdf59a1de8 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:41:50.409689 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1064/2111] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 1 16:41:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 16:41:50.411236 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ac86a8-af90-4bb1-a44b-e01981e64de9 None None] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:41:50.411551 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ac86a8-af90-4bb1-a44b-e01981e64de9 None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:41:50.411819 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ac86a8-af90-4bb1-a44b-e01981e64de9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:41:50.436223 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.auth.core [None req-b6ac86a8-af90-4bb1-a44b-e01981e64de9 None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80312) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:41:50.437449 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-b6ac86a8-af90-4bb1-a44b-e01981e64de9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:41:50.454833 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1065/2112] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 1 16:41:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 16:41:50.460677 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ac86a8-af90-4bb1-a44b-e01981e64de9 None None] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:41:50.469566 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d832c720-aa76-4388-a86c-36d15a8a6926 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:41:50.470909 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d832c720-aa76-4388-a86c-36d15a8a6926 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:41:50.471382 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d832c720-aa76-4388-a86c-36d15a8a6926 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:41:50.471877 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d832c720-aa76-4388-a86c-36d15a8a6926 admin admin] PATH_INFO: `/v3/projects/947c7f038ac44414a96a2f273cd74437` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:41:50.474577 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ac86a8-af90-4bb1-a44b-e01981e64de9 None None] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:41:50.475448 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d832c720-aa76-4388-a86c-36d15a8a6926 admin admin] RBAC: Authorizing `identity:delete_project(project_id=947c7f038ac44414a96a2f273cd74437)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:41:50.478370 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d832c720-aa76-4388-a86c-36d15a8a6926 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:41:50.479950 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92707653-b1f3-41ab-b6c9-67fb4132e335 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:41:50.480797 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92707653-b1f3-41ab-b6c9-67fb4132e335 admin admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:41:50.481152 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92707653-b1f3-41ab-b6c9-67fb4132e335 admin admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:41:50.481506 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92707653-b1f3-41ab-b6c9-67fb4132e335 admin admin] PATH_INFO: `/v3/domains` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:41:50.483281 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92707653-b1f3-41ab-b6c9-67fb4132e335 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80312) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:41:50.484652 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92707653-b1f3-41ab-b6c9-67fb4132e335 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:41:50.487692 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92707653-b1f3-41ab-b6c9-67fb4132e335 admin admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:41:50.502555 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1066/2113] 10.222.0.57 () {58 vars in 1152 bytes} [Fri Jul 1 16:41:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:41:50.508152 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92707653-b1f3-41ab-b6c9-67fb4132e335 admin admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:41:50.520737 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92707653-b1f3-41ab-b6c9-67fb4132e335 admin admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:41:50.527807 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fcbf8e1-558f-4327-ab7b-6d8ab07758c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:41:50.528938 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcbf8e1-558f-4327-ab7b-6d8ab07758c7 admin admin] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:41:50.529355 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcbf8e1-558f-4327-ab7b-6d8ab07758c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:41:50.531034 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcbf8e1-558f-4327-ab7b-6d8ab07758c7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:41:50.532957 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fcbf8e1-558f-4327-ab7b-6d8ab07758c7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:41:50.536605 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fcbf8e1-558f-4327-ab7b-6d8ab07758c7 admin admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:41:50.581873 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-d832c720-aa76-4388-a86c-36d15a8a6926 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '947c7f038ac44414a96a2f273cd74437', 'request_id': 'req-d832c720-aa76-4388-a86c-36d15a8a6926'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:41:50.595337 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1067/2114] 10.222.0.57 () {60 vars in 1152 bytes} [Fri Jul 1 16:41:50 2022] POST /identity/v3/projects => generated 373 bytes in 84 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 16:41:50.595337 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fcbf8e1-558f-4327-ab7b-6d8ab07758c7 admin admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:41:50.603199 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-5fcbf8e1-558f-4327-ab7b-6d8ab07758c7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:41:50.612523 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-d832c720-aa76-4388-a86c-36d15a8a6926 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '947c7f038ac44414a96a2f273cd74437', 'request_id': 'req-d832c720-aa76-4388-a86c-36d15a8a6926'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:41:50.619820 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-d832c720-aa76-4388-a86c-36d15a8a6926 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '947c7f038ac44414a96a2f273cd74437', 'request_id': 'req-d832c720-aa76-4388-a86c-36d15a8a6926'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:41:50.640432 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-d832c720-aa76-4388-a86c-36d15a8a6926 admin admin] The token cache is being invalidate because project 947c7f038ac44414a96a2f273cd74437 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80311) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 01 16:41:50.640799 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-d832c720-aa76-4388-a86c-36d15a8a6926 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:41:50.641440 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-d832c720-aa76-4388-a86c-36d15a8a6926 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:41:50.643571 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1048/2115] 10.222.0.57 () {58 vars in 1232 bytes} [Fri Jul 1 16:41:50 2022] DELETE /identity/v3/projects/947c7f038ac44414a96a2f273cd74437 => generated 0 bytes in 409 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 16:41:50.644184 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d832c720-aa76-4388-a86c-36d15a8a6926 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:41:50.645706 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-d832c720-aa76-4388-a86c-36d15a8a6926 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:41:50.649792 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:41:50.705940 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fcbf8e1-558f-4327-ab7b-6d8ab07758c7 admin admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:41:50.711421 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14547b8-64f1-413e-87b6-0118691e391f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:41:50.712630 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14547b8-64f1-413e-87b6-0118691e391f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:41:50.713090 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14547b8-64f1-413e-87b6-0118691e391f admin admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:41:50.713522 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14547b8-64f1-413e-87b6-0118691e391f admin admin] PATH_INFO: `/v3/projects/2c49e143df7c4d059430b7dc5bffd475` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:41:50.714961 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c14547b8-64f1-413e-87b6-0118691e391f admin admin] RBAC: Authorizing `identity:delete_project(project_id=2c49e143df7c4d059430b7dc5bffd475)` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:41:50.716871 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c14547b8-64f1-413e-87b6-0118691e391f admin admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:41:50.736300 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d832c720-aa76-4388-a86c-36d15a8a6926 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:41:50.743769 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3b733d-6c64-45ad-9f04-c79e282d3634 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:41:50.744700 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3b733d-6c64-45ad-9f04-c79e282d3634 admin admin] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:41:50.745133 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3b733d-6c64-45ad-9f04-c79e282d3634 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:41:50.745806 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3b733d-6c64-45ad-9f04-c79e282d3634 admin admin] PATH_INFO: `/v3/users` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:41:50.746551 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df3b733d-6c64-45ad-9f04-c79e282d3634 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:41:50.748513 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df3b733d-6c64-45ad-9f04-c79e282d3634 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:41:50.780270 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1049/2116] 10.222.0.57 () {60 vars in 1143 bytes} [Fri Jul 1 16:41:50 2022] POST /identity/v3/users => generated 287 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 16:41:50.782909 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4e71b7-86bd-4f29-8320-244d5eee4d1e None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:41:50.783422 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4e71b7-86bd-4f29-8320-244d5eee4d1e None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:41:50.784770 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4e71b7-86bd-4f29-8320-244d5eee4d1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:41:50.789711 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-c14547b8-64f1-413e-87b6-0118691e391f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2c49e143df7c4d059430b7dc5bffd475', 'request_id': 'req-c14547b8-64f1-413e-87b6-0118691e391f'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:41:50.801470 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-c14547b8-64f1-413e-87b6-0118691e391f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2c49e143df7c4d059430b7dc5bffd475', 'request_id': 'req-c14547b8-64f1-413e-87b6-0118691e391f'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:41:50.808826 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-c14547b8-64f1-413e-87b6-0118691e391f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2c49e143df7c4d059430b7dc5bffd475', 'request_id': 'req-c14547b8-64f1-413e-87b6-0118691e391f'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:41:50.810294 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-7e4e71b7-86bd-4f29-8320-244d5eee4d1e None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:41:50.812057 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-7e4e71b7-86bd-4f29-8320-244d5eee4d1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:41:50.824384 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-c14547b8-64f1-413e-87b6-0118691e391f admin admin] The token cache is being invalidate because project 2c49e143df7c4d059430b7dc5bffd475 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80312) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 01 16:41:50.824766 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-c14547b8-64f1-413e-87b6-0118691e391f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:41:50.825536 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-c14547b8-64f1-413e-87b6-0118691e391f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:41:50.827214 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1068/2117] 10.222.0.57 () {58 vars in 1232 bytes} [Fri Jul 1 16:41:50 2022] DELETE /identity/v3/projects/2c49e143df7c4d059430b7dc5bffd475 => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 16:41:50.827978 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14547b8-64f1-413e-87b6-0118691e391f admin admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:41:50.829965 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-c14547b8-64f1-413e-87b6-0118691e391f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:41:50.843110 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:41:50.923471 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1050/2118] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 1 16:41:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 16:41:50.925797 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b752b8-d522-4a35-b269-865076b77922 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:41:50.926463 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b752b8-d522-4a35-b269-865076b77922 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:41:50.927076 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b752b8-d522-4a35-b269-865076b77922 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:41:50.932478 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14547b8-64f1-413e-87b6-0118691e391f admin admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:41:50.940341 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f172b0e7-e673-4705-a7b4-dd16fbad42e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:41:50.941519 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f172b0e7-e673-4705-a7b4-dd16fbad42e3 admin admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:41:50.941715 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f172b0e7-e673-4705-a7b4-dd16fbad42e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:41:50.941948 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f172b0e7-e673-4705-a7b4-dd16fbad42e3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:41:50.947032 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f172b0e7-e673-4705-a7b4-dd16fbad42e3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:41:50.950999 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f172b0e7-e673-4705-a7b4-dd16fbad42e3 admin admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:41:50.956706 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-47b752b8-d522-4a35-b269-865076b77922 None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:41:50.957943 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-47b752b8-d522-4a35-b269-865076b77922 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:41:50.963508 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1069/2119] 10.222.0.57 () {58 vars in 1121 bytes} [Fri Jul 1 16:41:50 2022] GET /identity/v3/roles => generated 1420 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 16:41:50.965305 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e60b234-4ac4-4f21-9f05-f5402005ac3a None None] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:41:50.965761 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e60b234-4ac4-4f21-9f05-f5402005ac3a None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:41:50.966175 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e60b234-4ac4-4f21-9f05-f5402005ac3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:41:50.972700 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1051/2120] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 1 16:41:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 16:41:50.980469 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b752b8-d522-4a35-b269-865076b77922 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:41:50.987921 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.auth.core [None req-2e60b234-4ac4-4f21-9f05-f5402005ac3a None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80312) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:41:50.989270 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-2e60b234-4ac4-4f21-9f05-f5402005ac3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:41:50.993998 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b752b8-d522-4a35-b269-865076b77922 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:41:50.998536 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1070/2121] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 1 16:41:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 16:41:50.999880 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6eaa98-f6e4-4059-8ab9-5be84951ee07 None None] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:41:51.000671 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6eaa98-f6e4-4059-8ab9-5be84951ee07 None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:41:51.000955 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6eaa98-f6e4-4059-8ab9-5be84951ee07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:41:51.002821 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28c66ac-956a-47bd-8339-c3caf968e2ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:41:51.003572 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28c66ac-956a-47bd-8339-c3caf968e2ef admin admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:41:51.003931 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28c66ac-956a-47bd-8339-c3caf968e2ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:41:51.004284 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28c66ac-956a-47bd-8339-c3caf968e2ef admin admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:41:51.005120 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d28c66ac-956a-47bd-8339-c3caf968e2ef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:41:51.009367 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d28c66ac-956a-47bd-8339-c3caf968e2ef admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:41:51.021284 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1052/2122] 10.222.0.57 () {58 vars in 1121 bytes} [Fri Jul 1 16:41:50 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 16:41:51.022262 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28c66ac-956a-47bd-8339-c3caf968e2ef admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:41:51.025251 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.auth.core [None req-7d6eaa98-f6e4-4059-8ab9-5be84951ee07 None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80312) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:41:51.026655 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-7d6eaa98-f6e4-4059-8ab9-5be84951ee07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:41:51.034803 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28c66ac-956a-47bd-8339-c3caf968e2ef admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:41:51.039290 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1071/2123] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 1 16:41:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 16:41:51.040049 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5e126a-bbf1-4bc6-806a-fa3eaf0c4208 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:41:51.040196 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6eaa98-f6e4-4059-8ab9-5be84951ee07 None None] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:41:51.040945 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5e126a-bbf1-4bc6-806a-fa3eaf0c4208 admin admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:41:51.041289 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5e126a-bbf1-4bc6-806a-fa3eaf0c4208 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:41:51.041629 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5e126a-bbf1-4bc6-806a-fa3eaf0c4208 admin admin] PATH_INFO: `/v3/domains` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:41:51.042735 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f5e126a-bbf1-4bc6-806a-fa3eaf0c4208 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:41:51.043113 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f5e126a-bbf1-4bc6-806a-fa3eaf0c4208 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:41:51.045001 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f5e126a-bbf1-4bc6-806a-fa3eaf0c4208 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:41:51.052110 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6eaa98-f6e4-4059-8ab9-5be84951ee07 None None] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:41:51.058068 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb789a38-a51e-423a-8af6-83dfd2980401 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:41:51.058993 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb789a38-a51e-423a-8af6-83dfd2980401 admin admin] REQUEST_METHOD: `PUT` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:41:51.059273 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb789a38-a51e-423a-8af6-83dfd2980401 admin admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:41:51.059683 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1053/2124] 10.222.0.57 () {58 vars in 1152 bytes} [Fri Jul 1 16:41:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:41:51.060695 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5e126a-bbf1-4bc6-806a-fa3eaf0c4208 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:41:51.062729 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb789a38-a51e-423a-8af6-83dfd2980401 admin admin] PATH_INFO: `/v3/projects/b6598cf49d3a4426be411fa45be0c7c8/users/1f2a1540974a444f8341d395d6a2b070/roles/983be40bdab04c478eaa90c6b6a46b14` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:41:51.072416 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5e126a-bbf1-4bc6-806a-fa3eaf0c4208 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:41:51.077622 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba2f586-4875-4c61-b211-410995a2b55f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:41:51.078465 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba2f586-4875-4c61-b211-410995a2b55f admin admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:41:51.079640 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba2f586-4875-4c61-b211-410995a2b55f admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:41:51.080026 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba2f586-4875-4c61-b211-410995a2b55f admin admin] PATH_INFO: `/v3/domains` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:41:51.080843 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba2f586-4875-4c61-b211-410995a2b55f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:41:51.081283 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba2f586-4875-4c61-b211-410995a2b55f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:41:51.082951 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba2f586-4875-4c61-b211-410995a2b55f admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:41:51.084356 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb789a38-a51e-423a-8af6-83dfd2980401 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b6598cf49d3a4426be411fa45be0c7c8, user_id=1f2a1540974a444f8341d395d6a2b070, role_id=983be40bdab04c478eaa90c6b6a46b14)` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:41:51.087429 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb789a38-a51e-423a-8af6-83dfd2980401 admin admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:41:51.098786 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1054/2125] 10.222.0.57 () {58 vars in 1152 bytes} [Fri Jul 1 16:41:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:41:51.103367 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba2f586-4875-4c61-b211-410995a2b55f admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:41:51.105370 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1072/2126] 10.222.0.57 () {60 vars in 1482 bytes} [Fri Jul 1 16:41:51 2022] PUT /identity/v3/projects/b6598cf49d3a4426be411fa45be0c7c8/users/1f2a1540974a444f8341d395d6a2b070/roles/983be40bdab04c478eaa90c6b6a46b14 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 16:41:51.106201 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb789a38-a51e-423a-8af6-83dfd2980401 admin admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:41:51.121271 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba2f586-4875-4c61-b211-410995a2b55f admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:41:51.124929 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:41:51.195983 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb789a38-a51e-423a-8af6-83dfd2980401 admin admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:41:51.201546 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e29d34d-ff2d-4ec3-a90d-25d7c8b94a91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:41:51.202292 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e29d34d-ff2d-4ec3-a90d-25d7c8b94a91 admin admin] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:41:51.202715 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e29d34d-ff2d-4ec3-a90d-25d7c8b94a91 admin admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:41:51.203061 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e29d34d-ff2d-4ec3-a90d-25d7c8b94a91 admin admin] PATH_INFO: `/v3/projects` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:41:51.204273 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddbb0715-c8e7-4ed3-8e10-efa612bcae62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:41:51.204862 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e29d34d-ff2d-4ec3-a90d-25d7c8b94a91 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:41:51.205076 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbb0715-c8e7-4ed3-8e10-efa612bcae62 admin admin] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:41:51.205427 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbb0715-c8e7-4ed3-8e10-efa612bcae62 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:41:51.205784 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbb0715-c8e7-4ed3-8e10-efa612bcae62 admin admin] PATH_INFO: `/v3/projects` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:41:51.206579 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddbb0715-c8e7-4ed3-8e10-efa612bcae62 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:41:51.207029 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e29d34d-ff2d-4ec3-a90d-25d7c8b94a91 admin admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:41:51.210566 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddbb0715-c8e7-4ed3-8e10-efa612bcae62 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:41:51.247875 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1055/2127] 10.222.0.57 () {60 vars in 1152 bytes} [Fri Jul 1 16:41:51 2022] POST /identity/v3/projects => generated 383 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 16:41:51.247875 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2104560d-9de1-4edf-8f76-6aa4946b1d69 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:41:51.247875 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2104560d-9de1-4edf-8f76-6aa4946b1d69 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:41:51.247875 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2104560d-9de1-4edf-8f76-6aa4946b1d69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:41:51.254416 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1073/2128] 10.222.0.57 () {60 vars in 1152 bytes} [Fri Jul 1 16:41:51 2022] POST /identity/v3/projects => generated 373 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 16:41:51.258107 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e29d34d-ff2d-4ec3-a90d-25d7c8b94a91 admin admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:41:51.273535 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-2104560d-9de1-4edf-8f76-6aa4946b1d69 None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:41:51.275620 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-2104560d-9de1-4edf-8f76-6aa4946b1d69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:41:51.386753 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e29d34d-ff2d-4ec3-a90d-25d7c8b94a91 admin admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:41:51.392976 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1056/2129] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 1 16:41:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 16:41:51.393181 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2104560d-9de1-4edf-8f76-6aa4946b1d69 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:41:51.400135 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:41:51.401997 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d72cfa2-ff49-46bd-af96-7fe4b6226bde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:41:51.402834 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d72cfa2-ff49-46bd-af96-7fe4b6226bde admin admin] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:41:51.403120 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d72cfa2-ff49-46bd-af96-7fe4b6226bde admin admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:41:51.403399 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d72cfa2-ff49-46bd-af96-7fe4b6226bde admin admin] PATH_INFO: `/v3/users` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:41:51.404108 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d72cfa2-ff49-46bd-af96-7fe4b6226bde admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:41:51.406249 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d72cfa2-ff49-46bd-af96-7fe4b6226bde admin admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:41:51.412041 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2104560d-9de1-4edf-8f76-6aa4946b1d69 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:41:51.428666 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65789de-612d-4851-bd92-7d1029e68892 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:41:51.430074 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65789de-612d-4851-bd92-7d1029e68892 admin admin] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:41:51.430756 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65789de-612d-4851-bd92-7d1029e68892 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:41:51.431358 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65789de-612d-4851-bd92-7d1029e68892 admin admin] PATH_INFO: `/v3/users` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:41:51.432630 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65789de-612d-4851-bd92-7d1029e68892 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:41:51.439528 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65789de-612d-4851-bd92-7d1029e68892 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:41:51.459417 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1074/2130] 10.222.0.57 () {60 vars in 1143 bytes} [Fri Jul 1 16:41:51 2022] POST /identity/v3/users => generated 292 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 16:41:51.463217 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d72cfa2-ff49-46bd-af96-7fe4b6226bde admin admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:41:51.467645 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-4d72cfa2-ff49-46bd-af96-7fe4b6226bde admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:41:51.485135 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1057/2131] 10.222.0.57 () {60 vars in 1143 bytes} [Fri Jul 1 16:41:51 2022] POST /identity/v3/users => generated 287 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 16:41:51.486715 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65789de-612d-4851-bd92-7d1029e68892 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:41:51.505991 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65789de-612d-4851-bd92-7d1029e68892 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:41:51.514991 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961ef752-e054-4066-a540-110287bb9a48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:41:51.516357 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961ef752-e054-4066-a540-110287bb9a48 admin admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:41:51.516827 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961ef752-e054-4066-a540-110287bb9a48 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:41:51.517383 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961ef752-e054-4066-a540-110287bb9a48 admin admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:41:51.519008 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-961ef752-e054-4066-a540-110287bb9a48 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:41:51.522320 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-961ef752-e054-4066-a540-110287bb9a48 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:41:51.555189 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1058/2132] 10.222.0.57 () {58 vars in 1121 bytes} [Fri Jul 1 16:41:51 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 16:41:51.555189 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961ef752-e054-4066-a540-110287bb9a48 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:41:51.570201 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961ef752-e054-4066-a540-110287bb9a48 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:41:51.575968 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903ebe13-a3c5-4f5c-9f35-12b285de975c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:41:51.577019 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903ebe13-a3c5-4f5c-9f35-12b285de975c admin admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:41:51.577505 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903ebe13-a3c5-4f5c-9f35-12b285de975c admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:41:51.577927 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903ebe13-a3c5-4f5c-9f35-12b285de975c admin admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:41:51.578888 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-903ebe13-a3c5-4f5c-9f35-12b285de975c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:41:51.580824 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-903ebe13-a3c5-4f5c-9f35-12b285de975c admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:41:51.593185 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1059/2133] 10.222.0.57 () {58 vars in 1121 bytes} [Fri Jul 1 16:41:51 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 16:41:51.595209 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903ebe13-a3c5-4f5c-9f35-12b285de975c admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:41:51.598406 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d72cfa2-ff49-46bd-af96-7fe4b6226bde admin admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:41:51.604487 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8f7749-5204-4f09-99a0-e4b0f2f37ad3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ed200ce29684c5f80354fa9a8638fe2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0788e6725a84e57a4174284c1845755', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:41:51.605614 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8f7749-5204-4f09-99a0-e4b0f2f37ad3 service neutron] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:41:51.606058 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8f7749-5204-4f09-99a0-e4b0f2f37ad3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:41:51.607551 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8f7749-5204-4f09-99a0-e4b0f2f37ad3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:41:51.611653 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903ebe13-a3c5-4f5c-9f35-12b285de975c admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:41:51.618166 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e8bf39-9f49-45a9-a140-32f96eada76d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:41:51.618735 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e8bf39-9f49-45a9-a140-32f96eada76d admin admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:41:51.619016 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e8bf39-9f49-45a9-a140-32f96eada76d admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:41:51.619287 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e8bf39-9f49-45a9-a140-32f96eada76d admin admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:41:51.620251 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e8bf39-9f49-45a9-a140-32f96eada76d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:41:51.621846 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e8bf39-9f49-45a9-a140-32f96eada76d admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:41:51.623618 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af8f7749-5204-4f09-99a0-e4b0f2f37ad3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:41:51.626230 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af8f7749-5204-4f09-99a0-e4b0f2f37ad3 service neutron] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:41:51.633007 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1060/2134] 10.222.0.57 () {58 vars in 1121 bytes} [Fri Jul 1 16:41:51 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 16:41:51.636699 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e8bf39-9f49-45a9-a140-32f96eada76d admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:41:51.648135 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e8bf39-9f49-45a9-a140-32f96eada76d admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:41:51.649414 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1075/2135] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 1 16:41:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 189 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 16:41:51.651116 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8f7749-5204-4f09-99a0-e4b0f2f37ad3 service neutron] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:41:51.653511 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524cb08b-0abc-460d-af1d-9dd8bd0cce09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:41:51.654266 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524cb08b-0abc-460d-af1d-9dd8bd0cce09 admin admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:41:51.654653 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524cb08b-0abc-460d-af1d-9dd8bd0cce09 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:41:51.654987 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524cb08b-0abc-460d-af1d-9dd8bd0cce09 admin admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:41:51.655818 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-524cb08b-0abc-460d-af1d-9dd8bd0cce09 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:41:51.657594 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-524cb08b-0abc-460d-af1d-9dd8bd0cce09 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:41:51.670098 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1061/2136] 10.222.0.57 () {58 vars in 1121 bytes} [Fri Jul 1 16:41:51 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 16:41:51.678234 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524cb08b-0abc-460d-af1d-9dd8bd0cce09 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:41:51.684720 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8f7749-5204-4f09-99a0-e4b0f2f37ad3 service neutron] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:41:51.700117 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c3f597-23fc-4a49-a7ad-9236c926d8c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:41:51.704647 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c3f597-23fc-4a49-a7ad-9236c926d8c0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:41:51.705239 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524cb08b-0abc-460d-af1d-9dd8bd0cce09 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:41:51.705636 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c3f597-23fc-4a49-a7ad-9236c926d8c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:41:51.706697 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c3f597-23fc-4a49-a7ad-9236c926d8c0 admin admin] PATH_INFO: `/v3/projects/ad236b909912469c893c1e010fcfe6e2/users/b0a2c5fc458a447c8595000dfd4a2bd3/roles/983be40bdab04c478eaa90c6b6a46b14` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:41:51.715299 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0fe2121-0ee0-4d45-a4fd-3c0b5e292625 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:41:51.716203 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fe2121-0ee0-4d45-a4fd-3c0b5e292625 admin admin] REQUEST_METHOD: `PUT` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:41:51.716485 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fe2121-0ee0-4d45-a4fd-3c0b5e292625 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:41:51.718889 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fe2121-0ee0-4d45-a4fd-3c0b5e292625 admin admin] PATH_INFO: `/v3/projects/409e26052ea7472fb3448a97267cee95/users/ee49f8d0c0444d4fa4a6cb69f08f77a9/roles/983be40bdab04c478eaa90c6b6a46b14` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:41:51.751652 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0fe2121-0ee0-4d45-a4fd-3c0b5e292625 admin admin] RBAC: Authorizing `identity:create_grant(project_id=409e26052ea7472fb3448a97267cee95, user_id=ee49f8d0c0444d4fa4a6cb69f08f77a9, role_id=983be40bdab04c478eaa90c6b6a46b14)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:41:51.752955 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c3f597-23fc-4a49-a7ad-9236c926d8c0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ad236b909912469c893c1e010fcfe6e2, user_id=b0a2c5fc458a447c8595000dfd4a2bd3, role_id=983be40bdab04c478eaa90c6b6a46b14)` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:41:51.755586 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0fe2121-0ee0-4d45-a4fd-3c0b5e292625 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:41:51.760907 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c3f597-23fc-4a49-a7ad-9236c926d8c0 admin admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:41:51.798411 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1062/2137] 10.222.0.57 () {60 vars in 1482 bytes} [Fri Jul 1 16:41:51 2022] PUT /identity/v3/projects/409e26052ea7472fb3448a97267cee95/users/ee49f8d0c0444d4fa4a6cb69f08f77a9/roles/983be40bdab04c478eaa90c6b6a46b14 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 01 16:41:51.803439 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1076/2138] 10.222.0.57 () {60 vars in 1482 bytes} [Fri Jul 1 16:41:51 2022] PUT /identity/v3/projects/ad236b909912469c893c1e010fcfe6e2/users/b0a2c5fc458a447c8595000dfd4a2bd3/roles/983be40bdab04c478eaa90c6b6a46b14 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 16:41:51.807639 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271c49dd-32eb-4212-99ce-e8853c3555e0 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:41:51.810458 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271c49dd-32eb-4212-99ce-e8853c3555e0 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:41:51.814153 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271c49dd-32eb-4212-99ce-e8853c3555e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:41:51.814421 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67812744-bd7c-4bae-a5f6-1b96829004aa None None] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:41:51.814862 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67812744-bd7c-4bae-a5f6-1b96829004aa None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:41:51.815148 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67812744-bd7c-4bae-a5f6-1b96829004aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:41:51.830063 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:41:51.864442 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.auth.core [None req-67812744-bd7c-4bae-a5f6-1b96829004aa None None] MFA Rules not processed for user `b0a2c5fc458a447c8595000dfd4a2bd3`. Rule list: `[]` (Enabled: `True`). {{(pid=80312) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:41:51.866153 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-67812744-bd7c-4bae-a5f6-1b96829004aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:41:51.869295 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-271c49dd-32eb-4212-99ce-e8853c3555e0 None None] MFA Rules not processed for user `ee49f8d0c0444d4fa4a6cb69f08f77a9`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:41:51.871115 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-271c49dd-32eb-4212-99ce-e8853c3555e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:41:51.988873 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1077/2139] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 1 16:41:51 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 16:41:51.989861 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67812744-bd7c-4bae-a5f6-1b96829004aa None None] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:41:52.017699 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1063/2140] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 1 16:41:51 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 16:41:52.030580 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271c49dd-32eb-4212-99ce-e8853c3555e0 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:41:52.036202 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-271c49dd-32eb-4212-99ce-e8853c3555e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:41:52.130261 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67812744-bd7c-4bae-a5f6-1b96829004aa None None] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:41:52.143072 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad0fdcd-ee4d-490e-9587-4590331c42f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:41:52.143770 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad0fdcd-ee4d-490e-9587-4590331c42f4 admin admin] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:41:52.144060 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad0fdcd-ee4d-490e-9587-4590331c42f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:41:52.144343 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad0fdcd-ee4d-490e-9587-4590331c42f4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:41:52.145059 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad0fdcd-ee4d-490e-9587-4590331c42f4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:41:52.151369 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad0fdcd-ee4d-490e-9587-4590331c42f4 admin admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:41:52.177272 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:41:52.193289 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271c49dd-32eb-4212-99ce-e8853c3555e0 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:41:52.200803 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f9c08d-bcfb-47d6-b1b0-0355d297ffa7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a50f745b31941b7a7265c0fc852f48e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0788e6725a84e57a4174284c1845755', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:41:52.201857 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f9c08d-bcfb-47d6-b1b0-0355d297ffa7 service nova] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:41:52.202416 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f9c08d-bcfb-47d6-b1b0-0355d297ffa7 service nova] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:41:52.203240 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f9c08d-bcfb-47d6-b1b0-0355d297ffa7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:41:52.216263 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f9c08d-bcfb-47d6-b1b0-0355d297ffa7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:41:52.220451 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f9c08d-bcfb-47d6-b1b0-0355d297ffa7 service nova] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:41:52.224790 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1078/2141] 10.222.0.57 () {60 vars in 1152 bytes} [Fri Jul 1 16:41:51 2022] POST /identity/v3/projects => generated 383 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 16:41:52.226336 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad0fdcd-ee4d-490e-9587-4590331c42f4 admin admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:41:52.251689 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1064/2142] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 1 16:41:52 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 221 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 16:41:52.372134 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad0fdcd-ee4d-490e-9587-4590331c42f4 admin admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:41:52.378903 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1bd80d-f3d4-4cd9-a7ee-e32cc852bcf3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:41:52.379838 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1bd80d-f3d4-4cd9-a7ee-e32cc852bcf3 admin admin] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:41:52.380277 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1bd80d-f3d4-4cd9-a7ee-e32cc852bcf3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:41:52.381394 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1bd80d-f3d4-4cd9-a7ee-e32cc852bcf3 admin admin] PATH_INFO: `/v3/users` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:41:52.382255 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb1bd80d-f3d4-4cd9-a7ee-e32cc852bcf3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:41:52.385163 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb1bd80d-f3d4-4cd9-a7ee-e32cc852bcf3 admin admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:41:52.430711 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1079/2143] 10.222.0.57 () {60 vars in 1143 bytes} [Fri Jul 1 16:41:52 2022] POST /identity/v3/users => generated 298 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 01 16:41:52.434600 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f9c08d-bcfb-47d6-b1b0-0355d297ffa7 service nova] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:41:52.454038 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f9c08d-bcfb-47d6-b1b0-0355d297ffa7 service nova] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:41:52.464941 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae4f3c2-7f6a-4775-bfe1-04c7f2af09df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:41:52.468041 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae4f3c2-7f6a-4775-bfe1-04c7f2af09df admin admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:41:52.468499 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae4f3c2-7f6a-4775-bfe1-04c7f2af09df admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:41:52.468800 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae4f3c2-7f6a-4775-bfe1-04c7f2af09df admin admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:41:52.470320 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae4f3c2-7f6a-4775-bfe1-04c7f2af09df admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:41:52.475979 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae4f3c2-7f6a-4775-bfe1-04c7f2af09df admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:41:52.515825 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1065/2144] 10.222.0.57 () {58 vars in 1121 bytes} [Fri Jul 1 16:41:52 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 16:41:52.517997 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1bd80d-f3d4-4cd9-a7ee-e32cc852bcf3 admin admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:41:52.552199 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1bd80d-f3d4-4cd9-a7ee-e32cc852bcf3 admin admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:41:52.576014 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de41574-de39-4731-b501-7e3bd3d52b1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:41:52.582520 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de41574-de39-4731-b501-7e3bd3d52b1d admin admin] REQUEST_METHOD: `PUT` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:41:52.582973 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de41574-de39-4731-b501-7e3bd3d52b1d admin admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:41:52.583304 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de41574-de39-4731-b501-7e3bd3d52b1d admin admin] PATH_INFO: `/v3/projects/523c0b53f60d435f8531e8434e7a8e11/users/b1282a829c0b4e61946379efc3e2159b/roles/84ee739146934c1ab32c501ff8093644` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:41:52.592444 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:41:52.597201 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc03e69-2953-4069-b38c-be2fa385bf84 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:41:52.597988 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc03e69-2953-4069-b38c-be2fa385bf84 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:41:52.598281 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc03e69-2953-4069-b38c-be2fa385bf84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:41:52.638425 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de41574-de39-4731-b501-7e3bd3d52b1d admin admin] RBAC: Authorizing `identity:create_grant(project_id=523c0b53f60d435f8531e8434e7a8e11, user_id=b1282a829c0b4e61946379efc3e2159b, role_id=84ee739146934c1ab32c501ff8093644)` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:41:52.644964 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de41574-de39-4731-b501-7e3bd3d52b1d admin admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:41:52.654220 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-4bc03e69-2953-4069-b38c-be2fa385bf84 None None] MFA Rules not processed for user `1f2a1540974a444f8341d395d6a2b070`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:41:52.655578 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-4bc03e69-2953-4069-b38c-be2fa385bf84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:41:52.670252 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1080/2145] 10.222.0.57 () {60 vars in 1482 bytes} [Fri Jul 1 16:41:52 2022] PUT /identity/v3/projects/523c0b53f60d435f8531e8434e7a8e11/users/b1282a829c0b4e61946379efc3e2159b/roles/84ee739146934c1ab32c501ff8093644 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 16:41:52.675290 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de41574-de39-4731-b501-7e3bd3d52b1d admin admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:41:52.853233 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1066/2146] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 1 16:41:52 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 250 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 16:41:52.854268 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc03e69-2953-4069-b38c-be2fa385bf84 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:41:52.856055 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-4bc03e69-2953-4069-b38c-be2fa385bf84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:41:52.893420 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de41574-de39-4731-b501-7e3bd3d52b1d admin admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:41:52.911420 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f08c799-bd40-44f2-80d8-f22dbb287df2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:41:52.913065 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f08c799-bd40-44f2-80d8-f22dbb287df2 admin admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:41:52.913065 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f08c799-bd40-44f2-80d8-f22dbb287df2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:41:52.913065 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f08c799-bd40-44f2-80d8-f22dbb287df2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:41:52.913355 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f08c799-bd40-44f2-80d8-f22dbb287df2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:41:52.914977 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f08c799-bd40-44f2-80d8-f22dbb287df2 admin admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:41:52.962873 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1081/2147] 10.222.0.57 () {58 vars in 1121 bytes} [Fri Jul 1 16:41:52 2022] GET /identity/v3/roles => generated 1420 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 01 16:41:52.965219 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f08c799-bd40-44f2-80d8-f22dbb287df2 admin admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:41:52.997338 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f08c799-bd40-44f2-80d8-f22dbb287df2 admin admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:41:53.003367 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fbbe77-1f75-4834-b489-fa4f75fdd4e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:41:53.004359 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fbbe77-1f75-4834-b489-fa4f75fdd4e7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:41:53.004775 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fbbe77-1f75-4834-b489-fa4f75fdd4e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:41:53.005081 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fbbe77-1f75-4834-b489-fa4f75fdd4e7 admin admin] PATH_INFO: `/v3/projects/523c0b53f60d435f8531e8434e7a8e11/users/b1282a829c0b4e61946379efc3e2159b/roles/84ee739146934c1ab32c501ff8093644` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:41:53.008771 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0fbbe77-1f75-4834-b489-fa4f75fdd4e7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=523c0b53f60d435f8531e8434e7a8e11, user_id=b1282a829c0b4e61946379efc3e2159b, role_id=84ee739146934c1ab32c501ff8093644)` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:41:53.014762 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0fbbe77-1f75-4834-b489-fa4f75fdd4e7 admin admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:41:53.058394 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1082/2148] 10.222.0.57 () {60 vars in 1482 bytes} [Fri Jul 1 16:41:52 2022] PUT /identity/v3/projects/523c0b53f60d435f8531e8434e7a8e11/users/b1282a829c0b4e61946379efc3e2159b/roles/84ee739146934c1ab32c501ff8093644 => generated 0 bytes in 91 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 16:41:53.066077 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a0db49-c63b-4285-9957-d98892f699f0 None None] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:41:53.066704 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a0db49-c63b-4285-9957-d98892f699f0 None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:41:53.067323 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a0db49-c63b-4285-9957-d98892f699f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:41:53.093797 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc03e69-2953-4069-b38c-be2fa385bf84 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:41:53.143714 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.auth.core [None req-87a0db49-c63b-4285-9957-d98892f699f0 None None] MFA Rules not processed for user `b1282a829c0b4e61946379efc3e2159b`. Rule list: `[]` (Enabled: `True`). {{(pid=80312) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:41:53.144998 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-87a0db49-c63b-4285-9957-d98892f699f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:41:53.210901 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa751bde-c9f1-4751-ae37-17f2b94647ff service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbddf971a2354bbdbb9da1d63f3abf88', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0788e6725a84e57a4174284c1845755', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:41:53.211635 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa751bde-c9f1-4751-ae37-17f2b94647ff service cinder] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:41:53.211953 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa751bde-c9f1-4751-ae37-17f2b94647ff service cinder] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:41:53.212274 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa751bde-c9f1-4751-ae37-17f2b94647ff service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:41:53.272386 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:41:53.273058 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1083/2149] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 1 16:41:53 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 01 16:41:53.289520 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a0db49-c63b-4285-9957-d98892f699f0 None None] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:41:53.291201 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-87a0db49-c63b-4285-9957-d98892f699f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:41:53.299970 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa751bde-c9f1-4751-ae37-17f2b94647ff service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:41:53.301783 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa751bde-c9f1-4751-ae37-17f2b94647ff service cinder] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:41:53.304966 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a0db49-c63b-4285-9957-d98892f699f0 None None] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:41:53.309765 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-accd234a-98ef-4798-8d31-78fad1ce4890 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbddf971a2354bbdbb9da1d63f3abf88', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0788e6725a84e57a4174284c1845755', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:41:53.315233 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-accd234a-98ef-4798-8d31-78fad1ce4890 service cinder] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:41:53.315233 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-accd234a-98ef-4798-8d31-78fad1ce4890 service cinder] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:41:53.315233 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-accd234a-98ef-4798-8d31-78fad1ce4890 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:41:53.317156 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1067/2150] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 1 16:41:52 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 462 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 16:41:53.418864 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:41:53.433297 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-accd234a-98ef-4798-8d31-78fad1ce4890 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:41:53.435128 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-accd234a-98ef-4798-8d31-78fad1ce4890 service cinder] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:41:53.452199 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1084/2151] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 1 16:41:53 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 16:41:54.510046 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935a3000-5027-420f-b86f-438fbd6d8a61 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:41:54.510761 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935a3000-5027-420f-b86f-438fbd6d8a61 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:41:54.510761 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935a3000-5027-420f-b86f-438fbd6d8a61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:41:54.529380 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-935a3000-5027-420f-b86f-438fbd6d8a61 None None] MFA Rules not processed for user `c1ab0b52325c4ab387db034eb7dfcadb`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:41:54.530604 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-935a3000-5027-420f-b86f-438fbd6d8a61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:41:54.587151 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1068/2152] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 1 16:41:54 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 16:41:54.837253 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7634b95-4ebe-491b-a00b-dab70031e19f None None] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:41:54.837815 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7634b95-4ebe-491b-a00b-dab70031e19f None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:41:54.837815 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7634b95-4ebe-491b-a00b-dab70031e19f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:41:54.858075 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.auth.core [None req-f7634b95-4ebe-491b-a00b-dab70031e19f None None] MFA Rules not processed for user `1f2a1540974a444f8341d395d6a2b070`. Rule list: `[]` (Enabled: `True`). {{(pid=80312) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:41:54.859307 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-f7634b95-4ebe-491b-a00b-dab70031e19f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:41:54.866928 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1085/2153] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 1 16:41:54 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 01 16:41:55.747018 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935a3000-5027-420f-b86f-438fbd6d8a61 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:41:55.856718 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935a3000-5027-420f-b86f-438fbd6d8a61 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:41:55.865097 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5b9978-4531-4a2b-817e-6f0f3ca1ff5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:41:55.866317 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5b9978-4531-4a2b-817e-6f0f3ca1ff5f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:41:55.866858 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5b9978-4531-4a2b-817e-6f0f3ca1ff5f admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:41:55.867309 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5b9978-4531-4a2b-817e-6f0f3ca1ff5f admin admin] PATH_INFO: `/v3/users/ee49f8d0c0444d4fa4a6cb69f08f77a9` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:41:55.868959 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c5b9978-4531-4a2b-817e-6f0f3ca1ff5f admin admin] RBAC: Authorizing `identity:delete_user(user_id=ee49f8d0c0444d4fa4a6cb69f08f77a9)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:41:55.871951 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c5b9978-4531-4a2b-817e-6f0f3ca1ff5f admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:41:55.939069 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-7c5b9978-4531-4a2b-817e-6f0f3ca1ff5f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ee49f8d0c0444d4fa4a6cb69f08f77a9'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:41:55.939718 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-7c5b9978-4531-4a2b-817e-6f0f3ca1ff5f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ee49f8d0c0444d4fa4a6cb69f08f77a9'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:41:55.940315 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-7c5b9978-4531-4a2b-817e-6f0f3ca1ff5f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ee49f8d0c0444d4fa4a6cb69f08f77a9'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:41:55.948479 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-7c5b9978-4531-4a2b-817e-6f0f3ca1ff5f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ee49f8d0c0444d4fa4a6cb69f08f77a9'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:41:55.963493 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-7c5b9978-4531-4a2b-817e-6f0f3ca1ff5f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ee49f8d0c0444d4fa4a6cb69f08f77a9'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:41:55.984373 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1069/2154] 10.222.0.57 () {58 vars in 1223 bytes} [Fri Jul 1 16:41:55 2022] DELETE /identity/v3/users/ee49f8d0c0444d4fa4a6cb69f08f77a9 => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 16:41:55.990224 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59f8dd5-1f5b-43cb-8a95-fd76fec9dc41 None None] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:41:55.990863 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59f8dd5-1f5b-43cb-8a95-fd76fec9dc41 None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:41:55.990971 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59f8dd5-1f5b-43cb-8a95-fd76fec9dc41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:41:56.011392 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.auth.core [None req-e59f8dd5-1f5b-43cb-8a95-fd76fec9dc41 None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80312) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:41:56.012750 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-e59f8dd5-1f5b-43cb-8a95-fd76fec9dc41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:41:56.094597 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1086/2155] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 1 16:41:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 16:41:56.105956 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5b9978-4531-4a2b-817e-6f0f3ca1ff5f admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:41:56.107459 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-7c5b9978-4531-4a2b-817e-6f0f3ca1ff5f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:41:56.115571 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:41:56.177795 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5b9978-4531-4a2b-817e-6f0f3ca1ff5f admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:41:56.181606 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533c4fd5-ee08-45a7-ba2f-37b0ff2fa053 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ed200ce29684c5f80354fa9a8638fe2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0788e6725a84e57a4174284c1845755', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:41:56.182264 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533c4fd5-ee08-45a7-ba2f-37b0ff2fa053 service neutron] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:41:56.182584 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533c4fd5-ee08-45a7-ba2f-37b0ff2fa053 service neutron] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:41:56.182868 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533c4fd5-ee08-45a7-ba2f-37b0ff2fa053 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:41:56.193632 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533c4fd5-ee08-45a7-ba2f-37b0ff2fa053 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:41:56.195125 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533c4fd5-ee08-45a7-ba2f-37b0ff2fa053 service neutron] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:41:56.207547 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1070/2156] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 1 16:41:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 16:41:56.531902 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59f8dd5-1f5b-43cb-8a95-fd76fec9dc41 None None] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:41:56.533548 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-e59f8dd5-1f5b-43cb-8a95-fd76fec9dc41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:41:56.552907 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59f8dd5-1f5b-43cb-8a95-fd76fec9dc41 None None] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:41:56.558462 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6385e2d0-a4fa-43ad-9a99-0815b68f951e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:41:56.559320 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6385e2d0-a4fa-43ad-9a99-0815b68f951e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:41:56.559608 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6385e2d0-a4fa-43ad-9a99-0815b68f951e admin admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:41:56.559893 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6385e2d0-a4fa-43ad-9a99-0815b68f951e admin admin] PATH_INFO: `/v3/projects/409e26052ea7472fb3448a97267cee95` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:41:56.561217 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6385e2d0-a4fa-43ad-9a99-0815b68f951e admin admin] RBAC: Authorizing `identity:delete_project(project_id=409e26052ea7472fb3448a97267cee95)` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:41:56.570742 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6385e2d0-a4fa-43ad-9a99-0815b68f951e admin admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:41:56.621482 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-6385e2d0-a4fa-43ad-9a99-0815b68f951e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '409e26052ea7472fb3448a97267cee95', 'request_id': 'req-6385e2d0-a4fa-43ad-9a99-0815b68f951e'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:41:56.629309 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-6385e2d0-a4fa-43ad-9a99-0815b68f951e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '409e26052ea7472fb3448a97267cee95', 'request_id': 'req-6385e2d0-a4fa-43ad-9a99-0815b68f951e'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:41:56.636363 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-6385e2d0-a4fa-43ad-9a99-0815b68f951e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '409e26052ea7472fb3448a97267cee95', 'request_id': 'req-6385e2d0-a4fa-43ad-9a99-0815b68f951e'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:41:56.648467 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-6385e2d0-a4fa-43ad-9a99-0815b68f951e admin admin] The token cache is being invalidate because project 409e26052ea7472fb3448a97267cee95 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80312) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 01 16:41:56.648757 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-6385e2d0-a4fa-43ad-9a99-0815b68f951e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:41:56.649538 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-6385e2d0-a4fa-43ad-9a99-0815b68f951e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:41:56.651235 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1087/2157] 10.222.0.57 () {58 vars in 1232 bytes} [Fri Jul 1 16:41:56 2022] DELETE /identity/v3/projects/409e26052ea7472fb3448a97267cee95 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 01 16:41:56.656990 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db19d1d-aee7-46a5-afb3-51c6c44f3404 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:41:56.657343 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db19d1d-aee7-46a5-afb3-51c6c44f3404 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:41:56.657733 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db19d1d-aee7-46a5-afb3-51c6c44f3404 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:41:56.677109 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-6db19d1d-aee7-46a5-afb3-51c6c44f3404 None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:41:56.678363 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-6db19d1d-aee7-46a5-afb3-51c6c44f3404 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:41:56.744764 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1071/2158] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 1 16:41:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 16:41:56.752800 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eecaface-9e81-418e-916b-bd00bff657bd None None] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:41:56.753092 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eecaface-9e81-418e-916b-bd00bff657bd None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:41:56.753370 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eecaface-9e81-418e-916b-bd00bff657bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:41:56.772946 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.auth.core [None req-eecaface-9e81-418e-916b-bd00bff657bd None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80312) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:41:56.775007 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-eecaface-9e81-418e-916b-bd00bff657bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:41:56.786405 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1088/2159] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 1 16:41:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 16:41:56.789763 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db19d1d-aee7-46a5-afb3-51c6c44f3404 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:41:56.801640 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db19d1d-aee7-46a5-afb3-51c6c44f3404 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:41:56.807669 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ceb475-65ba-49f8-85c1-3bae40bfc32d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:41:56.808344 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ceb475-65ba-49f8-85c1-3bae40bfc32d admin admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:41:56.808634 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ceb475-65ba-49f8-85c1-3bae40bfc32d admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:41:56.808916 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ceb475-65ba-49f8-85c1-3bae40bfc32d admin admin] PATH_INFO: `/v3/domains` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:41:56.809663 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ceb475-65ba-49f8-85c1-3bae40bfc32d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:41:56.810314 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ceb475-65ba-49f8-85c1-3bae40bfc32d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:41:56.811972 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ceb475-65ba-49f8-85c1-3bae40bfc32d admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:41:56.831687 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1072/2160] 10.222.0.57 () {58 vars in 1152 bytes} [Fri Jul 1 16:41:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:41:56.835608 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eecaface-9e81-418e-916b-bd00bff657bd None None] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:41:56.855329 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eecaface-9e81-418e-916b-bd00bff657bd None None] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:41:56.862230 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52aea3cf-2a73-47cd-a2c1-c6c540ad501d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:41:56.862975 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52aea3cf-2a73-47cd-a2c1-c6c540ad501d admin admin] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:41:56.863325 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52aea3cf-2a73-47cd-a2c1-c6c540ad501d admin admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:41:56.863570 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52aea3cf-2a73-47cd-a2c1-c6c540ad501d admin admin] PATH_INFO: `/v3/projects` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:41:56.864272 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52aea3cf-2a73-47cd-a2c1-c6c540ad501d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:41:56.866246 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52aea3cf-2a73-47cd-a2c1-c6c540ad501d admin admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:41:56.895361 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1089/2161] 10.222.0.57 () {60 vars in 1152 bytes} [Fri Jul 1 16:41:56 2022] POST /identity/v3/projects => generated 383 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 16:41:56.899552 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ceb475-65ba-49f8-85c1-3bae40bfc32d admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:41:56.910213 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ceb475-65ba-49f8-85c1-3bae40bfc32d admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:41:56.996132 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de4b6db-9a54-44b5-bbde-ca77b26b90c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:41:56.996838 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de4b6db-9a54-44b5-bbde-ca77b26b90c0 admin admin] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:41:56.997121 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de4b6db-9a54-44b5-bbde-ca77b26b90c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:41:56.997400 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de4b6db-9a54-44b5-bbde-ca77b26b90c0 admin admin] PATH_INFO: `/v3/users` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:41:56.998122 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de4b6db-9a54-44b5-bbde-ca77b26b90c0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:41:57.000106 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de4b6db-9a54-44b5-bbde-ca77b26b90c0 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:41:57.025659 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1073/2162] 10.222.0.57 () {60 vars in 1143 bytes} [Fri Jul 1 16:41:56 2022] POST /identity/v3/users => generated 292 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 01 16:41:57.030476 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52aea3cf-2a73-47cd-a2c1-c6c540ad501d admin admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:41:57.045453 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52aea3cf-2a73-47cd-a2c1-c6c540ad501d admin admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:41:57.051494 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695954db-e4f6-4113-9b37-bf9653625a86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:41:57.052306 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695954db-e4f6-4113-9b37-bf9653625a86 admin admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:41:57.052626 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695954db-e4f6-4113-9b37-bf9653625a86 admin admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:41:57.053689 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695954db-e4f6-4113-9b37-bf9653625a86 admin admin] PATH_INFO: `/v3/roles` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:41:57.054477 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695954db-e4f6-4113-9b37-bf9653625a86 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:41:57.056370 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695954db-e4f6-4113-9b37-bf9653625a86 admin admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:41:57.067829 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1090/2163] 10.222.0.57 () {58 vars in 1121 bytes} [Fri Jul 1 16:41:57 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 16:41:57.072325 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de4b6db-9a54-44b5-bbde-ca77b26b90c0 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:41:57.083625 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de4b6db-9a54-44b5-bbde-ca77b26b90c0 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:41:57.088899 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079f52a9-6892-4b40-8e0a-79ca94c75c1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:41:57.089571 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079f52a9-6892-4b40-8e0a-79ca94c75c1c admin admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:41:57.089890 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079f52a9-6892-4b40-8e0a-79ca94c75c1c admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:41:57.090556 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079f52a9-6892-4b40-8e0a-79ca94c75c1c admin admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:41:57.091042 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-079f52a9-6892-4b40-8e0a-79ca94c75c1c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:41:57.092774 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-079f52a9-6892-4b40-8e0a-79ca94c75c1c admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:41:57.107546 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1074/2164] 10.222.0.57 () {58 vars in 1121 bytes} [Fri Jul 1 16:41:57 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 16:41:57.112068 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695954db-e4f6-4113-9b37-bf9653625a86 admin admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:41:57.122648 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695954db-e4f6-4113-9b37-bf9653625a86 admin admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:41:57.127687 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221eac8e-0d14-4153-8a91-575d655439eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:41:57.128553 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221eac8e-0d14-4153-8a91-575d655439eb admin admin] REQUEST_METHOD: `PUT` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:41:57.128832 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221eac8e-0d14-4153-8a91-575d655439eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:41:57.129107 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221eac8e-0d14-4153-8a91-575d655439eb admin admin] PATH_INFO: `/v3/projects/dd47a19119b24b8ca99d7e5aaa84b4a6/users/15906c7dd1dc4cf2a7006d3de72348a6/roles/983be40bdab04c478eaa90c6b6a46b14` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:41:57.145745 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-221eac8e-0d14-4153-8a91-575d655439eb admin admin] RBAC: Authorizing `identity:create_grant(project_id=dd47a19119b24b8ca99d7e5aaa84b4a6, user_id=15906c7dd1dc4cf2a7006d3de72348a6, role_id=983be40bdab04c478eaa90c6b6a46b14)` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:41:57.148609 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-221eac8e-0d14-4153-8a91-575d655439eb admin admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:41:57.164451 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1091/2165] 10.222.0.57 () {60 vars in 1482 bytes} [Fri Jul 1 16:41:57 2022] PUT /identity/v3/projects/dd47a19119b24b8ca99d7e5aaa84b4a6/users/15906c7dd1dc4cf2a7006d3de72348a6/roles/983be40bdab04c478eaa90c6b6a46b14 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 01 16:41:57.176240 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c1d480-dce2-4c2b-830f-a28e7746ff97 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:41:57.176677 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c1d480-dce2-4c2b-830f-a28e7746ff97 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:41:57.176988 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c1d480-dce2-4c2b-830f-a28e7746ff97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:41:57.200918 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:41:57.205043 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-18c1d480-dce2-4c2b-830f-a28e7746ff97 None None] MFA Rules not processed for user `15906c7dd1dc4cf2a7006d3de72348a6`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:41:57.206989 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-18c1d480-dce2-4c2b-830f-a28e7746ff97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:41:57.274378 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1075/2166] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 1 16:41:57 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 16:41:57.278919 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221eac8e-0d14-4153-8a91-575d655439eb admin admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:41:57.287541 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:41:57.369687 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221eac8e-0d14-4153-8a91-575d655439eb admin admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:41:57.376179 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519ec9c5-5b50-4327-9a1d-892cc0600890 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:41:57.376988 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519ec9c5-5b50-4327-9a1d-892cc0600890 admin admin] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:41:57.377339 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519ec9c5-5b50-4327-9a1d-892cc0600890 admin admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:41:57.377717 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519ec9c5-5b50-4327-9a1d-892cc0600890 admin admin] PATH_INFO: `/v3/projects` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:41:57.378647 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519ec9c5-5b50-4327-9a1d-892cc0600890 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:41:57.380957 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519ec9c5-5b50-4327-9a1d-892cc0600890 admin admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:41:57.418477 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1092/2167] 10.222.0.57 () {60 vars in 1152 bytes} [Fri Jul 1 16:41:57 2022] POST /identity/v3/projects => generated 383 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 16:41:57.419923 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c1d480-dce2-4c2b-830f-a28e7746ff97 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:41:57.432768 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c1d480-dce2-4c2b-830f-a28e7746ff97 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:41:57.517336 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67aa7f8-ee20-466a-98f2-885583d9f1a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:41:57.518139 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67aa7f8-ee20-466a-98f2-885583d9f1a3 admin admin] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:41:57.518468 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67aa7f8-ee20-466a-98f2-885583d9f1a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:41:57.518748 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67aa7f8-ee20-466a-98f2-885583d9f1a3 admin admin] PATH_INFO: `/v3/users` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:41:57.519445 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b67aa7f8-ee20-466a-98f2-885583d9f1a3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:41:57.521627 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b67aa7f8-ee20-466a-98f2-885583d9f1a3 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:41:57.545757 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1076/2168] 10.222.0.57 () {60 vars in 1143 bytes} [Fri Jul 1 16:41:57 2022] POST /identity/v3/users => generated 298 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 16:41:57.550682 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519ec9c5-5b50-4327-9a1d-892cc0600890 admin admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:41:57.562641 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519ec9c5-5b50-4327-9a1d-892cc0600890 admin admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:41:57.567750 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84fc7cb9-f143-4696-b705-d030329af653 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:41:57.568431 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fc7cb9-f143-4696-b705-d030329af653 admin admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:41:57.568860 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fc7cb9-f143-4696-b705-d030329af653 admin admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:41:57.569150 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fc7cb9-f143-4696-b705-d030329af653 admin admin] PATH_INFO: `/v3/roles` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:41:57.569905 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84fc7cb9-f143-4696-b705-d030329af653 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:41:57.571725 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84fc7cb9-f143-4696-b705-d030329af653 admin admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:41:57.583512 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1093/2169] 10.222.0.57 () {58 vars in 1121 bytes} [Fri Jul 1 16:41:57 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 16:41:57.588641 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67aa7f8-ee20-466a-98f2-885583d9f1a3 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:41:57.600247 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67aa7f8-ee20-466a-98f2-885583d9f1a3 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:41:57.605738 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f37accd-2663-471a-91bc-25af091b2c97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:41:57.606781 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f37accd-2663-471a-91bc-25af091b2c97 admin admin] REQUEST_METHOD: `PUT` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:41:57.607033 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f37accd-2663-471a-91bc-25af091b2c97 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:41:57.607305 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f37accd-2663-471a-91bc-25af091b2c97 admin admin] PATH_INFO: `/v3/projects/fccdd994e25a481ca6e0d06d2d9ff387/users/65984a6fa0404acb8280b98ab5ebe13e/roles/84ee739146934c1ab32c501ff8093644` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:41:57.624139 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f37accd-2663-471a-91bc-25af091b2c97 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fccdd994e25a481ca6e0d06d2d9ff387, user_id=65984a6fa0404acb8280b98ab5ebe13e, role_id=84ee739146934c1ab32c501ff8093644)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:41:57.626925 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f37accd-2663-471a-91bc-25af091b2c97 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:41:57.640551 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1077/2170] 10.222.0.57 () {60 vars in 1482 bytes} [Fri Jul 1 16:41:57 2022] PUT /identity/v3/projects/fccdd994e25a481ca6e0d06d2d9ff387/users/65984a6fa0404acb8280b98ab5ebe13e/roles/84ee739146934c1ab32c501ff8093644 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 16:41:57.645107 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84fc7cb9-f143-4696-b705-d030329af653 admin admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:41:57.655010 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84fc7cb9-f143-4696-b705-d030329af653 admin admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:41:57.712513 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d86242-6b5f-49ea-9ead-2d8d9d16765d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:41:57.713076 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d86242-6b5f-49ea-9ead-2d8d9d16765d admin admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:41:57.713360 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d86242-6b5f-49ea-9ead-2d8d9d16765d admin admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:41:57.713638 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d86242-6b5f-49ea-9ead-2d8d9d16765d admin admin] PATH_INFO: `/v3/roles` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:41:57.714386 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d86242-6b5f-49ea-9ead-2d8d9d16765d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:41:57.715996 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d86242-6b5f-49ea-9ead-2d8d9d16765d admin admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:41:57.726441 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1094/2171] 10.222.0.57 () {58 vars in 1121 bytes} [Fri Jul 1 16:41:57 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 01 16:41:57.730598 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f37accd-2663-471a-91bc-25af091b2c97 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:41:57.741347 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f37accd-2663-471a-91bc-25af091b2c97 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:41:57.746156 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26e81b6-7da6-4f9a-985a-3791393f5f14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:41:57.747063 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26e81b6-7da6-4f9a-985a-3791393f5f14 admin admin] REQUEST_METHOD: `PUT` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:41:57.747348 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26e81b6-7da6-4f9a-985a-3791393f5f14 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:41:57.747629 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26e81b6-7da6-4f9a-985a-3791393f5f14 admin admin] PATH_INFO: `/v3/projects/fccdd994e25a481ca6e0d06d2d9ff387/users/65984a6fa0404acb8280b98ab5ebe13e/roles/84ee739146934c1ab32c501ff8093644` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:41:57.749583 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26e81b6-7da6-4f9a-985a-3791393f5f14 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fccdd994e25a481ca6e0d06d2d9ff387, user_id=65984a6fa0404acb8280b98ab5ebe13e, role_id=84ee739146934c1ab32c501ff8093644)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:41:57.753495 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26e81b6-7da6-4f9a-985a-3791393f5f14 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:41:57.763895 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1078/2172] 10.222.0.57 () {60 vars in 1482 bytes} [Fri Jul 1 16:41:57 2022] PUT /identity/v3/projects/fccdd994e25a481ca6e0d06d2d9ff387/users/65984a6fa0404acb8280b98ab5ebe13e/roles/84ee739146934c1ab32c501ff8093644 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 16:41:57.774853 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a121b33-d00d-4f0b-8e2b-29b334bb2a55 None None] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:41:57.775147 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a121b33-d00d-4f0b-8e2b-29b334bb2a55 None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:41:57.775431 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a121b33-d00d-4f0b-8e2b-29b334bb2a55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:41:57.803621 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.auth.core [None req-4a121b33-d00d-4f0b-8e2b-29b334bb2a55 None None] MFA Rules not processed for user `65984a6fa0404acb8280b98ab5ebe13e`. Rule list: `[]` (Enabled: `True`). {{(pid=80312) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:41:57.804834 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-4a121b33-d00d-4f0b-8e2b-29b334bb2a55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:41:57.904139 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:41:57.910448 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1095/2173] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 1 16:41:57 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 16:41:57.932900 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26e81b6-7da6-4f9a-985a-3791393f5f14 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:41:57.935006 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-d26e81b6-7da6-4f9a-985a-3791393f5f14 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:41:57.995601 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a121b33-d00d-4f0b-8e2b-29b334bb2a55 None None] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:41:57.997301 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-4a121b33-d00d-4f0b-8e2b-29b334bb2a55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:41:58.027272 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26e81b6-7da6-4f9a-985a-3791393f5f14 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:41:58.033160 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f42e7be-d383-42b9-b972-211c78bc8383 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbddf971a2354bbdbb9da1d63f3abf88', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0788e6725a84e57a4174284c1845755', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:41:58.044882 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f42e7be-d383-42b9-b972-211c78bc8383 service cinder] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:41:58.045449 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f42e7be-d383-42b9-b972-211c78bc8383 service cinder] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:41:58.045847 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f42e7be-d383-42b9-b972-211c78bc8383 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:41:58.072393 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f42e7be-d383-42b9-b972-211c78bc8383 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:41:58.078621 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f42e7be-d383-42b9-b972-211c78bc8383 service cinder] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:41:58.084934 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:41:58.102944 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1079/2174] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 1 16:41:57 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 16:41:58.118149 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148aa68a-9a2a-4513-a903-101905828ba0 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:41:58.118627 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148aa68a-9a2a-4513-a903-101905828ba0 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:41:58.119035 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148aa68a-9a2a-4513-a903-101905828ba0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:41:58.128691 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a121b33-d00d-4f0b-8e2b-29b334bb2a55 None None] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:41:58.146903 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9e353e-2681-457f-9102-9fb144dcfdc2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbddf971a2354bbdbb9da1d63f3abf88', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0788e6725a84e57a4174284c1845755', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:41:58.148451 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9e353e-2681-457f-9102-9fb144dcfdc2 service cinder] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:41:58.149215 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9e353e-2681-457f-9102-9fb144dcfdc2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:41:58.152737 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9e353e-2681-457f-9102-9fb144dcfdc2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:41:58.154034 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-148aa68a-9a2a-4513-a903-101905828ba0 None None] MFA Rules not processed for user `c4de1b1eeaed4fc5ae7200b2c7a90957`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:41:58.155609 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-148aa68a-9a2a-4513-a903-101905828ba0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:41:58.163698 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-1b9e353e-2681-457f-9102-9fb144dcfdc2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:41:58.237745 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1080/2175] 10.222.0.57 () {58 vars in 1009 bytes} [Fri Jul 1 16:41:58 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 16:41:58.254312 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148aa68a-9a2a-4513-a903-101905828ba0 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:41:58.255989 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-148aa68a-9a2a-4513-a903-101905828ba0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:41:58.358771 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9e353e-2681-457f-9102-9fb144dcfdc2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:41:58.360970 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9e353e-2681-457f-9102-9fb144dcfdc2 service cinder] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:41:58.377146 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1096/2176] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 1 16:41:57 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 384 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 01 16:41:58.379188 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9e353e-2681-457f-9102-9fb144dcfdc2 service cinder] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:41:58.398572 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9e353e-2681-457f-9102-9fb144dcfdc2 service cinder] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:41:58.400265 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148aa68a-9a2a-4513-a903-101905828ba0 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:41:58.407329 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646ee89d-dd5b-4903-992d-6b2e005be34f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '859fe410c2d2450fb803c6244839f27c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0788e6725a84e57a4174284c1845755', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:41:58.407988 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646ee89d-dd5b-4903-992d-6b2e005be34f service swift] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:41:58.408281 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646ee89d-dd5b-4903-992d-6b2e005be34f service swift] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:41:58.408565 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646ee89d-dd5b-4903-992d-6b2e005be34f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:41:58.413341 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb76147-d656-40b1-a77d-8a0882186dbd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbddf971a2354bbdbb9da1d63f3abf88', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0788e6725a84e57a4174284c1845755', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:41:58.414012 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb76147-d656-40b1-a77d-8a0882186dbd service cinder] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:41:58.414311 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb76147-d656-40b1-a77d-8a0882186dbd service cinder] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:41:58.414637 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb76147-d656-40b1-a77d-8a0882186dbd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:41:58.420095 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-646ee89d-dd5b-4903-992d-6b2e005be34f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:41:58.423105 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-646ee89d-dd5b-4903-992d-6b2e005be34f service swift] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:41:58.437296 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1081/2177] 10.222.0.57 () {60 vars in 1384 bytes} [Fri Jul 1 16:41:58 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 191 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 01 16:41:58.562246 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb76147-d656-40b1-a77d-8a0882186dbd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:41:58.566523 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb76147-d656-40b1-a77d-8a0882186dbd service cinder] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:41:58.594962 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1097/2178] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 1 16:41:58 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 217 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 16:42:01.975656 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646ee89d-dd5b-4903-992d-6b2e005be34f service swift] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:42:02.068382 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646ee89d-dd5b-4903-992d-6b2e005be34f service swift] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:42:02.073870 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0ebec4-72f9-49b1-b8ee-af511dc791aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:42:02.074948 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0ebec4-72f9-49b1-b8ee-af511dc791aa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:42:02.075230 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0ebec4-72f9-49b1-b8ee-af511dc791aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:42:02.075507 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0ebec4-72f9-49b1-b8ee-af511dc791aa admin admin] PATH_INFO: `/v3/users/15906c7dd1dc4cf2a7006d3de72348a6` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:42:02.076731 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb0ebec4-72f9-49b1-b8ee-af511dc791aa admin admin] RBAC: Authorizing `identity:delete_user(user_id=15906c7dd1dc4cf2a7006d3de72348a6)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:42:02.080802 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb0ebec4-72f9-49b1-b8ee-af511dc791aa admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:42:02.138881 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:42:02.150427 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-eb0ebec4-72f9-49b1-b8ee-af511dc791aa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '15906c7dd1dc4cf2a7006d3de72348a6'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:42:02.151978 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-eb0ebec4-72f9-49b1-b8ee-af511dc791aa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '15906c7dd1dc4cf2a7006d3de72348a6'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:42:02.152979 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-eb0ebec4-72f9-49b1-b8ee-af511dc791aa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '15906c7dd1dc4cf2a7006d3de72348a6'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:42:02.163112 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-eb0ebec4-72f9-49b1-b8ee-af511dc791aa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '15906c7dd1dc4cf2a7006d3de72348a6'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:42:02.182413 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-eb0ebec4-72f9-49b1-b8ee-af511dc791aa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '15906c7dd1dc4cf2a7006d3de72348a6'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:42:02.201387 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1082/2179] 10.222.0.57 () {58 vars in 1223 bytes} [Fri Jul 1 16:42:01 2022] DELETE /identity/v3/users/15906c7dd1dc4cf2a7006d3de72348a6 => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 16:42:02.203175 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa5f259-8cd7-4c98-9c03-b72bac7d2d0d None None] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:42:02.207868 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa5f259-8cd7-4c98-9c03-b72bac7d2d0d None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:42:02.207868 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa5f259-8cd7-4c98-9c03-b72bac7d2d0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:42:02.237138 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.auth.core [None req-9aa5f259-8cd7-4c98-9c03-b72bac7d2d0d None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80312) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:42:02.238925 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-9aa5f259-8cd7-4c98-9c03-b72bac7d2d0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:42:02.315530 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=100 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:42:02.343331 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1098/2180] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 1 16:42:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 01 16:42:02.354533 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0ebec4-72f9-49b1-b8ee-af511dc791aa admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:42:02.356271 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-eb0ebec4-72f9-49b1-b8ee-af511dc791aa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:42:02.418865 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0ebec4-72f9-49b1-b8ee-af511dc791aa admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:42:02.423049 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ad3631-f3b0-4736-aeff-34174f3fc1f2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ed200ce29684c5f80354fa9a8638fe2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0788e6725a84e57a4174284c1845755', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:42:02.423721 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ad3631-f3b0-4736-aeff-34174f3fc1f2 service neutron] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:42:02.424005 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ad3631-f3b0-4736-aeff-34174f3fc1f2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:42:02.424286 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ad3631-f3b0-4736-aeff-34174f3fc1f2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:42:02.434158 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ad3631-f3b0-4736-aeff-34174f3fc1f2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:42:02.435634 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ad3631-f3b0-4736-aeff-34174f3fc1f2 service neutron] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:42:02.447296 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1083/2181] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 1 16:42:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 16:42:02.814133 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa5f259-8cd7-4c98-9c03-b72bac7d2d0d None None] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:42:02.816167 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-9aa5f259-8cd7-4c98-9c03-b72bac7d2d0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:42:02.829892 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa5f259-8cd7-4c98-9c03-b72bac7d2d0d None None] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:42:02.838390 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6495d87b-121c-46ee-8a0d-514818d498e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:42:02.838390 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6495d87b-121c-46ee-8a0d-514818d498e3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:42:02.838390 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6495d87b-121c-46ee-8a0d-514818d498e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:42:02.838390 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6495d87b-121c-46ee-8a0d-514818d498e3 admin admin] PATH_INFO: `/v3/projects/dd47a19119b24b8ca99d7e5aaa84b4a6` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:42:02.838814 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6495d87b-121c-46ee-8a0d-514818d498e3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dd47a19119b24b8ca99d7e5aaa84b4a6)` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:42:02.839550 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6495d87b-121c-46ee-8a0d-514818d498e3 admin admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:42:02.938203 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-6495d87b-121c-46ee-8a0d-514818d498e3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dd47a19119b24b8ca99d7e5aaa84b4a6', 'request_id': 'req-6495d87b-121c-46ee-8a0d-514818d498e3'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:42:02.947236 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-6495d87b-121c-46ee-8a0d-514818d498e3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dd47a19119b24b8ca99d7e5aaa84b4a6', 'request_id': 'req-6495d87b-121c-46ee-8a0d-514818d498e3'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:42:02.953738 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-6495d87b-121c-46ee-8a0d-514818d498e3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dd47a19119b24b8ca99d7e5aaa84b4a6', 'request_id': 'req-6495d87b-121c-46ee-8a0d-514818d498e3'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:42:02.974812 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-6495d87b-121c-46ee-8a0d-514818d498e3 admin admin] The token cache is being invalidate because project dd47a19119b24b8ca99d7e5aaa84b4a6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80312) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 01 16:42:02.975500 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-6495d87b-121c-46ee-8a0d-514818d498e3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:42:02.977984 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-6495d87b-121c-46ee-8a0d-514818d498e3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:42:02.983448 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1099/2182] 10.222.0.57 () {58 vars in 1232 bytes} [Fri Jul 1 16:42:02 2022] DELETE /identity/v3/projects/dd47a19119b24b8ca99d7e5aaa84b4a6 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 16:42:02.989381 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ad3631-f3b0-4736-aeff-34174f3fc1f2 service neutron] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:42:03.006728 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ad3631-f3b0-4736-aeff-34174f3fc1f2 service neutron] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:42:03.114879 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae7eaed-ebd2-4d3f-a39c-000080de85f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:42:03.116293 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae7eaed-ebd2-4d3f-a39c-000080de85f7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:42:03.116872 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae7eaed-ebd2-4d3f-a39c-000080de85f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:42:03.117310 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae7eaed-ebd2-4d3f-a39c-000080de85f7 admin admin] PATH_INFO: `/v3/users/65984a6fa0404acb8280b98ab5ebe13e` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:42:03.118928 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae7eaed-ebd2-4d3f-a39c-000080de85f7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=65984a6fa0404acb8280b98ab5ebe13e)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:42:03.122034 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae7eaed-ebd2-4d3f-a39c-000080de85f7 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:42:03.190155 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-cae7eaed-ebd2-4d3f-a39c-000080de85f7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '65984a6fa0404acb8280b98ab5ebe13e'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:42:03.190912 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-cae7eaed-ebd2-4d3f-a39c-000080de85f7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '65984a6fa0404acb8280b98ab5ebe13e'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:42:03.191515 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-cae7eaed-ebd2-4d3f-a39c-000080de85f7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '65984a6fa0404acb8280b98ab5ebe13e'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:42:03.202536 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-cae7eaed-ebd2-4d3f-a39c-000080de85f7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '65984a6fa0404acb8280b98ab5ebe13e'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:42:03.209120 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:42:03.228505 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-cae7eaed-ebd2-4d3f-a39c-000080de85f7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '65984a6fa0404acb8280b98ab5ebe13e'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:42:03.262734 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1084/2183] 10.222.0.57 () {58 vars in 1223 bytes} [Fri Jul 1 16:42:02 2022] DELETE /identity/v3/users/65984a6fa0404acb8280b98ab5ebe13e => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 16:42:03.634692 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6495d87b-121c-46ee-8a0d-514818d498e3 admin admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:42:03.636179 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-6495d87b-121c-46ee-8a0d-514818d498e3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:42:03.667819 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6495d87b-121c-46ee-8a0d-514818d498e3 admin admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:42:03.758931 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49821dc3-3641-4ccb-94b0-9067b9193781 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:42:03.760069 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49821dc3-3641-4ccb-94b0-9067b9193781 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:42:03.760355 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49821dc3-3641-4ccb-94b0-9067b9193781 admin admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:42:03.760630 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49821dc3-3641-4ccb-94b0-9067b9193781 admin admin] PATH_INFO: `/v3/projects/fccdd994e25a481ca6e0d06d2d9ff387` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:42:03.762078 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49821dc3-3641-4ccb-94b0-9067b9193781 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fccdd994e25a481ca6e0d06d2d9ff387)` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:42:03.764059 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49821dc3-3641-4ccb-94b0-9067b9193781 admin admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:42:03.824649 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-49821dc3-3641-4ccb-94b0-9067b9193781 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fccdd994e25a481ca6e0d06d2d9ff387', 'request_id': 'req-49821dc3-3641-4ccb-94b0-9067b9193781'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:42:03.833736 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-49821dc3-3641-4ccb-94b0-9067b9193781 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fccdd994e25a481ca6e0d06d2d9ff387', 'request_id': 'req-49821dc3-3641-4ccb-94b0-9067b9193781'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:42:03.845483 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-49821dc3-3641-4ccb-94b0-9067b9193781 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fccdd994e25a481ca6e0d06d2d9ff387', 'request_id': 'req-49821dc3-3641-4ccb-94b0-9067b9193781'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:42:03.865373 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-49821dc3-3641-4ccb-94b0-9067b9193781 admin admin] The token cache is being invalidate because project fccdd994e25a481ca6e0d06d2d9ff387 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80312) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 01 16:42:03.865785 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-49821dc3-3641-4ccb-94b0-9067b9193781 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:42:03.866621 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-49821dc3-3641-4ccb-94b0-9067b9193781 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:42:03.868318 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1100/2184] 10.222.0.57 () {58 vars in 1232 bytes} [Fri Jul 1 16:42:03 2022] DELETE /identity/v3/projects/fccdd994e25a481ca6e0d06d2d9ff387 => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 16:42:03.875767 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0455f5b-ce22-4456-9bda-91b81e94a3d7 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:42:03.876297 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0455f5b-ce22-4456-9bda-91b81e94a3d7 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:42:03.877077 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0455f5b-ce22-4456-9bda-91b81e94a3d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:42:03.904215 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-a0455f5b-ce22-4456-9bda-91b81e94a3d7 None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:42:03.905425 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-a0455f5b-ce22-4456-9bda-91b81e94a3d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:42:03.994931 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1085/2185] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 1 16:42:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 16:42:04.007975 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a117674d-523f-419f-907f-cccf38259677 None None] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:42:04.008421 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a117674d-523f-419f-907f-cccf38259677 None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:42:04.008878 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a117674d-523f-419f-907f-cccf38259677 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:42:04.017301 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:42:04.034301 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.auth.core [None req-a117674d-523f-419f-907f-cccf38259677 None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80312) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:42:04.035658 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-a117674d-523f-419f-907f-cccf38259677 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:42:04.049905 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0455f5b-ce22-4456-9bda-91b81e94a3d7 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:42:04.054651 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1101/2186] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 1 16:42:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 16:42:04.061047 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0455f5b-ce22-4456-9bda-91b81e94a3d7 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:42:04.066519 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76e1e26-0e1f-4123-bd62-8a94adfd25bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:42:04.067238 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76e1e26-0e1f-4123-bd62-8a94adfd25bd admin admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:42:04.067558 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76e1e26-0e1f-4123-bd62-8a94adfd25bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:42:04.067842 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76e1e26-0e1f-4123-bd62-8a94adfd25bd admin admin] PATH_INFO: `/v3/domains` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:42:04.068646 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d76e1e26-0e1f-4123-bd62-8a94adfd25bd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:42:04.068980 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d76e1e26-0e1f-4123-bd62-8a94adfd25bd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:42:04.070663 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d76e1e26-0e1f-4123-bd62-8a94adfd25bd admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:42:04.102487 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1086/2187] 10.222.0.57 () {58 vars in 1152 bytes} [Fri Jul 1 16:42:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:42:04.105772 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a117674d-523f-419f-907f-cccf38259677 None None] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:42:04.140377 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a117674d-523f-419f-907f-cccf38259677 None None] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:42:04.157056 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb631cf6-7cc9-4d04-9173-64040e31327e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:42:04.158245 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb631cf6-7cc9-4d04-9173-64040e31327e admin admin] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:42:04.158620 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb631cf6-7cc9-4d04-9173-64040e31327e admin admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:42:04.158906 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb631cf6-7cc9-4d04-9173-64040e31327e admin admin] PATH_INFO: `/v3/projects` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:42:04.161344 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb631cf6-7cc9-4d04-9173-64040e31327e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:42:04.169536 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb631cf6-7cc9-4d04-9173-64040e31327e admin admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:42:04.183415 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76e1e26-0e1f-4123-bd62-8a94adfd25bd admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:42:04.185674 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-d76e1e26-0e1f-4123-bd62-8a94adfd25bd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:42:04.205291 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76e1e26-0e1f-4123-bd62-8a94adfd25bd admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:42:04.219295 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d743d1-3186-4438-9c77-446034254235 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:42:04.220240 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d743d1-3186-4438-9c77-446034254235 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:42:04.220507 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d743d1-3186-4438-9c77-446034254235 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:42:04.220903 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d743d1-3186-4438-9c77-446034254235 admin admin] PATH_INFO: `/v3/users/b0a2c5fc458a447c8595000dfd4a2bd3` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:42:04.222239 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d743d1-3186-4438-9c77-446034254235 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b0a2c5fc458a447c8595000dfd4a2bd3)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:42:04.224504 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d743d1-3186-4438-9c77-446034254235 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:42:04.235374 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1102/2188] 10.222.0.57 () {60 vars in 1152 bytes} [Fri Jul 1 16:42:04 2022] POST /identity/v3/projects => generated 379 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 16:42:04.239640 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb631cf6-7cc9-4d04-9173-64040e31327e admin admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:42:04.311074 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-11d743d1-3186-4438-9c77-446034254235 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b0a2c5fc458a447c8595000dfd4a2bd3'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:42:04.313945 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-11d743d1-3186-4438-9c77-446034254235 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b0a2c5fc458a447c8595000dfd4a2bd3'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:42:04.318830 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-11d743d1-3186-4438-9c77-446034254235 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b0a2c5fc458a447c8595000dfd4a2bd3'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:42:04.335842 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-11d743d1-3186-4438-9c77-446034254235 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b0a2c5fc458a447c8595000dfd4a2bd3'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:42:04.368132 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-11d743d1-3186-4438-9c77-446034254235 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b0a2c5fc458a447c8595000dfd4a2bd3'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:42:04.399470 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1087/2189] 10.222.0.57 () {58 vars in 1223 bytes} [Fri Jul 1 16:42:04 2022] DELETE /identity/v3/users/b0a2c5fc458a447c8595000dfd4a2bd3 => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 16:42:04.403703 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47eebcfb-9c74-460d-a2e8-fc03f7f15cff None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:42:04.403992 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47eebcfb-9c74-460d-a2e8-fc03f7f15cff None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:42:04.404332 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47eebcfb-9c74-460d-a2e8-fc03f7f15cff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:42:04.419333 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb631cf6-7cc9-4d04-9173-64040e31327e admin admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:42:04.446228 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-47eebcfb-9c74-460d-a2e8-fc03f7f15cff None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:42:04.449537 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-47eebcfb-9c74-460d-a2e8-fc03f7f15cff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:42:04.527078 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222ca937-7a36-4330-999a-50d28ac01c76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:42:04.527793 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222ca937-7a36-4330-999a-50d28ac01c76 admin admin] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:42:04.528083 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222ca937-7a36-4330-999a-50d28ac01c76 admin admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:42:04.528363 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222ca937-7a36-4330-999a-50d28ac01c76 admin admin] PATH_INFO: `/v3/users` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:42:04.529096 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-222ca937-7a36-4330-999a-50d28ac01c76 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:42:04.533134 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:42:04.538102 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-222ca937-7a36-4330-999a-50d28ac01c76 admin admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:42:04.586472 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1103/2190] 10.222.0.57 () {60 vars in 1143 bytes} [Fri Jul 1 16:42:04 2022] POST /identity/v3/users => generated 290 bytes in 347 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 16:42:04.587496 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222ca937-7a36-4330-999a-50d28ac01c76 admin admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:42:04.590709 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-222ca937-7a36-4330-999a-50d28ac01c76 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:42:04.608241 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222ca937-7a36-4330-999a-50d28ac01c76 admin admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:42:04.609035 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1088/2191] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 1 16:42:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 01 16:42:04.623444 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47eebcfb-9c74-460d-a2e8-fc03f7f15cff None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:42:04.625407 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-47eebcfb-9c74-460d-a2e8-fc03f7f15cff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:42:04.632431 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65815cde-0694-4565-8464-2c994326b63f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:42:04.634029 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65815cde-0694-4565-8464-2c994326b63f admin admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:42:04.634029 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65815cde-0694-4565-8464-2c994326b63f admin admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:42:04.634280 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65815cde-0694-4565-8464-2c994326b63f admin admin] PATH_INFO: `/v3/roles` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:42:04.635944 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65815cde-0694-4565-8464-2c994326b63f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:42:04.645592 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65815cde-0694-4565-8464-2c994326b63f admin admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:42:04.666297 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1104/2192] 10.222.0.57 () {58 vars in 1121 bytes} [Fri Jul 1 16:42:04 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 16:42:04.670814 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65815cde-0694-4565-8464-2c994326b63f admin admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:42:04.682232 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65815cde-0694-4565-8464-2c994326b63f admin admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:42:04.688149 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94a0c4d-02cf-4f77-8f9d-b821053a4c66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:42:04.688831 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94a0c4d-02cf-4f77-8f9d-b821053a4c66 admin admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:42:04.689114 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94a0c4d-02cf-4f77-8f9d-b821053a4c66 admin admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:42:04.689387 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94a0c4d-02cf-4f77-8f9d-b821053a4c66 admin admin] PATH_INFO: `/v3/roles` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:42:04.690286 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e94a0c4d-02cf-4f77-8f9d-b821053a4c66 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:42:04.692870 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e94a0c4d-02cf-4f77-8f9d-b821053a4c66 admin admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:42:04.714524 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1105/2193] 10.222.0.57 () {58 vars in 1121 bytes} [Fri Jul 1 16:42:04 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 16:42:04.717560 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94a0c4d-02cf-4f77-8f9d-b821053a4c66 admin admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:42:04.718485 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47eebcfb-9c74-460d-a2e8-fc03f7f15cff None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:42:04.723998 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96e1b9c-ff67-4c4d-a05e-d7775cdb55a8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ed200ce29684c5f80354fa9a8638fe2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0788e6725a84e57a4174284c1845755', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:42:04.725011 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96e1b9c-ff67-4c4d-a05e-d7775cdb55a8 service neutron] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:42:04.725452 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96e1b9c-ff67-4c4d-a05e-d7775cdb55a8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:42:04.725905 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96e1b9c-ff67-4c4d-a05e-d7775cdb55a8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:42:04.734100 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94a0c4d-02cf-4f77-8f9d-b821053a4c66 admin admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:42:04.738676 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f96e1b9c-ff67-4c4d-a05e-d7775cdb55a8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:42:04.740549 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f96e1b9c-ff67-4c4d-a05e-d7775cdb55a8 service neutron] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:42:04.740831 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9dec45-f464-49c9-8809-fbfb4cdfc68d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:42:04.741930 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9dec45-f464-49c9-8809-fbfb4cdfc68d admin admin] REQUEST_METHOD: `PUT` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:42:04.742223 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9dec45-f464-49c9-8809-fbfb4cdfc68d admin admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:42:04.742528 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9dec45-f464-49c9-8809-fbfb4cdfc68d admin admin] PATH_INFO: `/v3/projects/46b020c27df14f6885dd3daf5271c2f0/users/fc2248eef607473d84be9cb3e06c3f35/roles/983be40bdab04c478eaa90c6b6a46b14` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:42:04.760650 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1089/2194] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 1 16:42:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 01 16:42:04.795847 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b9dec45-f464-49c9-8809-fbfb4cdfc68d admin admin] RBAC: Authorizing `identity:create_grant(project_id=46b020c27df14f6885dd3daf5271c2f0, user_id=fc2248eef607473d84be9cb3e06c3f35, role_id=983be40bdab04c478eaa90c6b6a46b14)` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:42:04.799375 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b9dec45-f464-49c9-8809-fbfb4cdfc68d admin admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:42:04.833474 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1106/2195] 10.222.0.57 () {60 vars in 1482 bytes} [Fri Jul 1 16:42:04 2022] PUT /identity/v3/projects/46b020c27df14f6885dd3daf5271c2f0/users/fc2248eef607473d84be9cb3e06c3f35/roles/983be40bdab04c478eaa90c6b6a46b14 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 16:42:04.840222 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a186966-84a7-4006-b5e5-4976f30ed670 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:42:04.840707 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a186966-84a7-4006-b5e5-4976f30ed670 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:42:04.841179 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a186966-84a7-4006-b5e5-4976f30ed670 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:42:04.900710 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-1a186966-84a7-4006-b5e5-4976f30ed670 None None] MFA Rules not processed for user `fc2248eef607473d84be9cb3e06c3f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:42:04.902245 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-1a186966-84a7-4006-b5e5-4976f30ed670 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:42:05.028251 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1090/2196] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 1 16:42:04 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 16:42:05.030297 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9dec45-f464-49c9-8809-fbfb4cdfc68d admin admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:42:05.071704 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:42:05.129284 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9dec45-f464-49c9-8809-fbfb4cdfc68d admin admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:42:05.134307 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235d6528-7a21-49d4-bfd8-a93d51a84c1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:42:05.135883 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235d6528-7a21-49d4-bfd8-a93d51a84c1f admin admin] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:42:05.136028 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235d6528-7a21-49d4-bfd8-a93d51a84c1f admin admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:42:05.136206 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235d6528-7a21-49d4-bfd8-a93d51a84c1f admin admin] PATH_INFO: `/v3/projects` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:42:05.136907 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-235d6528-7a21-49d4-bfd8-a93d51a84c1f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:42:05.139026 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-235d6528-7a21-49d4-bfd8-a93d51a84c1f admin admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:42:05.179729 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a186966-84a7-4006-b5e5-4976f30ed670 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:42:05.181740 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-1a186966-84a7-4006-b5e5-4976f30ed670 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:42:05.185299 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1107/2197] 10.222.0.57 () {60 vars in 1152 bytes} [Fri Jul 1 16:42:05 2022] POST /identity/v3/projects => generated 381 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 16:42:05.186413 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235d6528-7a21-49d4-bfd8-a93d51a84c1f admin admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:42:05.197695 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a186966-84a7-4006-b5e5-4976f30ed670 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:42:05.317561 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bcb5b22-32b1-4949-9d54-e54021f55315 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:42:05.318886 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235d6528-7a21-49d4-bfd8-a93d51a84c1f admin admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:42:05.319158 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcb5b22-32b1-4949-9d54-e54021f55315 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:42:05.319522 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcb5b22-32b1-4949-9d54-e54021f55315 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:42:05.320892 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcb5b22-32b1-4949-9d54-e54021f55315 admin admin] PATH_INFO: `/v3/projects/ad236b909912469c893c1e010fcfe6e2` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:42:05.322565 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bcb5b22-32b1-4949-9d54-e54021f55315 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ad236b909912469c893c1e010fcfe6e2)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:42:05.325004 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bcb5b22-32b1-4949-9d54-e54021f55315 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:42:05.338251 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:42:05.339406 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c6a7e9-3c24-4fbd-8651-ad7583398fe2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:42:05.340533 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c6a7e9-3c24-4fbd-8651-ad7583398fe2 admin admin] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:42:05.340983 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c6a7e9-3c24-4fbd-8651-ad7583398fe2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:42:05.343064 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c6a7e9-3c24-4fbd-8651-ad7583398fe2 admin admin] PATH_INFO: `/v3/users` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:42:05.345802 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c6a7e9-3c24-4fbd-8651-ad7583398fe2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:42:05.348350 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c6a7e9-3c24-4fbd-8651-ad7583398fe2 admin admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:42:05.389197 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1108/2198] 10.222.0.57 () {60 vars in 1143 bytes} [Fri Jul 1 16:42:05 2022] POST /identity/v3/users => generated 297 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 16:42:05.393519 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c6a7e9-3c24-4fbd-8651-ad7583398fe2 admin admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:42:05.409920 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c6a7e9-3c24-4fbd-8651-ad7583398fe2 admin admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:42:05.415412 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6358299-713d-4598-9155-5eb5bee9574e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:42:05.416148 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6358299-713d-4598-9155-5eb5bee9574e admin admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:42:05.416852 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6358299-713d-4598-9155-5eb5bee9574e admin admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:42:05.417146 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6358299-713d-4598-9155-5eb5bee9574e admin admin] PATH_INFO: `/v3/roles` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:42:05.417977 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6358299-713d-4598-9155-5eb5bee9574e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:42:05.419872 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6358299-713d-4598-9155-5eb5bee9574e admin admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:42:05.434514 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1109/2199] 10.222.0.57 () {58 vars in 1121 bytes} [Fri Jul 1 16:42:05 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 16:42:05.439344 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6358299-713d-4598-9155-5eb5bee9574e admin admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:42:05.441028 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-3bcb5b22-32b1-4949-9d54-e54021f55315 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ad236b909912469c893c1e010fcfe6e2', 'request_id': 'req-3bcb5b22-32b1-4949-9d54-e54021f55315'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:42:05.448759 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-3bcb5b22-32b1-4949-9d54-e54021f55315 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ad236b909912469c893c1e010fcfe6e2', 'request_id': 'req-3bcb5b22-32b1-4949-9d54-e54021f55315'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:42:05.450672 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6358299-713d-4598-9155-5eb5bee9574e admin admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:42:05.462058 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-3bcb5b22-32b1-4949-9d54-e54021f55315 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ad236b909912469c893c1e010fcfe6e2', 'request_id': 'req-3bcb5b22-32b1-4949-9d54-e54021f55315'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:42:05.479678 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-3bcb5b22-32b1-4949-9d54-e54021f55315 admin admin] The token cache is being invalidate because project ad236b909912469c893c1e010fcfe6e2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80311) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 01 16:42:05.480103 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-3bcb5b22-32b1-4949-9d54-e54021f55315 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:42:05.482387 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-3bcb5b22-32b1-4949-9d54-e54021f55315 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:42:05.488170 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1091/2200] 10.222.0.57 () {58 vars in 1232 bytes} [Fri Jul 1 16:42:05 2022] DELETE /identity/v3/projects/ad236b909912469c893c1e010fcfe6e2 => generated 0 bytes in 307 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 16:42:05.490082 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bcb5b22-32b1-4949-9d54-e54021f55315 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:42:05.496502 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-3bcb5b22-32b1-4949-9d54-e54021f55315 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:42:05.567618 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03f7d1c-9a90-43bf-817a-40ee3821c3c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:42:05.571097 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03f7d1c-9a90-43bf-817a-40ee3821c3c9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:42:05.571097 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03f7d1c-9a90-43bf-817a-40ee3821c3c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:42:05.571097 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03f7d1c-9a90-43bf-817a-40ee3821c3c9 admin admin] PATH_INFO: `/v3/projects/f381929ffd944e5f8d19d9beaed7d9bf/users/17a8ba7ee4ab4a348bf000cd654f5c89/roles/84ee739146934c1ab32c501ff8093644` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:42:05.608630 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c03f7d1c-9a90-43bf-817a-40ee3821c3c9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f381929ffd944e5f8d19d9beaed7d9bf, user_id=17a8ba7ee4ab4a348bf000cd654f5c89, role_id=84ee739146934c1ab32c501ff8093644)` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:42:05.615473 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c03f7d1c-9a90-43bf-817a-40ee3821c3c9 admin admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:42:05.621266 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bcb5b22-32b1-4949-9d54-e54021f55315 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:42:05.628705 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beaf1a96-c549-40de-9333-e6cbe827ea18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:42:05.630199 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beaf1a96-c549-40de-9333-e6cbe827ea18 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:42:05.630732 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beaf1a96-c549-40de-9333-e6cbe827ea18 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:42:05.631257 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beaf1a96-c549-40de-9333-e6cbe827ea18 admin admin] PATH_INFO: `/v3/users/b1282a829c0b4e61946379efc3e2159b` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:42:05.634244 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beaf1a96-c549-40de-9333-e6cbe827ea18 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b1282a829c0b4e61946379efc3e2159b)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:42:05.636500 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beaf1a96-c549-40de-9333-e6cbe827ea18 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:42:05.643051 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1110/2201] 10.222.0.57 () {60 vars in 1482 bytes} [Fri Jul 1 16:42:05 2022] PUT /identity/v3/projects/f381929ffd944e5f8d19d9beaed7d9bf/users/17a8ba7ee4ab4a348bf000cd654f5c89/roles/84ee739146934c1ab32c501ff8093644 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 16:42:05.649318 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03f7d1c-9a90-43bf-817a-40ee3821c3c9 admin admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:42:05.651899 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-c03f7d1c-9a90-43bf-817a-40ee3821c3c9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:42:05.670572 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:42:05.746571 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-beaf1a96-c549-40de-9333-e6cbe827ea18 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b1282a829c0b4e61946379efc3e2159b'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:42:05.748547 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-beaf1a96-c549-40de-9333-e6cbe827ea18 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b1282a829c0b4e61946379efc3e2159b'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:42:05.749238 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-beaf1a96-c549-40de-9333-e6cbe827ea18 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b1282a829c0b4e61946379efc3e2159b'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:42:05.779341 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-beaf1a96-c549-40de-9333-e6cbe827ea18 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b1282a829c0b4e61946379efc3e2159b'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:42:05.790261 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03f7d1c-9a90-43bf-817a-40ee3821c3c9 admin admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:42:05.795281 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b8ef8e-9f1f-4c40-93db-6b5aff015e48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:42:05.795974 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b8ef8e-9f1f-4c40-93db-6b5aff015e48 admin admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:42:05.796266 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b8ef8e-9f1f-4c40-93db-6b5aff015e48 admin admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:42:05.796552 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b8ef8e-9f1f-4c40-93db-6b5aff015e48 admin admin] PATH_INFO: `/v3/roles` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:42:05.797390 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b8ef8e-9f1f-4c40-93db-6b5aff015e48 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:42:05.799537 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b8ef8e-9f1f-4c40-93db-6b5aff015e48 admin admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:42:05.803732 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-beaf1a96-c549-40de-9333-e6cbe827ea18 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b1282a829c0b4e61946379efc3e2159b'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:42:05.818722 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1111/2202] 10.222.0.57 () {58 vars in 1121 bytes} [Fri Jul 1 16:42:05 2022] GET /identity/v3/roles => generated 1420 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 16:42:05.819722 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b8ef8e-9f1f-4c40-93db-6b5aff015e48 admin admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:42:05.821470 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-78b8ef8e-9f1f-4c40-93db-6b5aff015e48 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:42:05.835666 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1092/2203] 10.222.0.57 () {58 vars in 1223 bytes} [Fri Jul 1 16:42:05 2022] DELETE /identity/v3/users/b1282a829c0b4e61946379efc3e2159b => generated 0 bytes in 345 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 16:42:05.853962 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b8ef8e-9f1f-4c40-93db-6b5aff015e48 admin admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:42:05.939791 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da559054-7894-4fa9-a983-1517878091bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:42:05.943708 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da559054-7894-4fa9-a983-1517878091bd admin admin] REQUEST_METHOD: `PUT` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:42:05.944308 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da559054-7894-4fa9-a983-1517878091bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:42:05.944638 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da559054-7894-4fa9-a983-1517878091bd admin admin] PATH_INFO: `/v3/projects/f381929ffd944e5f8d19d9beaed7d9bf/users/17a8ba7ee4ab4a348bf000cd654f5c89/roles/84ee739146934c1ab32c501ff8093644` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:42:05.946878 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da559054-7894-4fa9-a983-1517878091bd admin admin] RBAC: Authorizing `identity:create_grant(project_id=f381929ffd944e5f8d19d9beaed7d9bf, user_id=17a8ba7ee4ab4a348bf000cd654f5c89, role_id=84ee739146934c1ab32c501ff8093644)` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:42:05.949622 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da559054-7894-4fa9-a983-1517878091bd admin admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:42:05.963975 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1112/2204] 10.222.0.57 () {60 vars in 1482 bytes} [Fri Jul 1 16:42:05 2022] PUT /identity/v3/projects/f381929ffd944e5f8d19d9beaed7d9bf/users/17a8ba7ee4ab4a348bf000cd654f5c89/roles/84ee739146934c1ab32c501ff8093644 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 16:42:05.974900 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5775729f-ec88-4312-9be5-12b1645c511f None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:42:05.975177 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5775729f-ec88-4312-9be5-12b1645c511f None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:42:05.975516 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5775729f-ec88-4312-9be5-12b1645c511f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:42:06.014875 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-5775729f-ec88-4312-9be5-12b1645c511f None None] MFA Rules not processed for user `17a8ba7ee4ab4a348bf000cd654f5c89`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:42:06.016909 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-5775729f-ec88-4312-9be5-12b1645c511f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:42:06.021574 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:42:06.150672 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da559054-7894-4fa9-a983-1517878091bd admin admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:42:06.151517 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1093/2205] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 1 16:42:05 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 16:42:06.152768 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-da559054-7894-4fa9-a983-1517878091bd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:42:06.236031 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da559054-7894-4fa9-a983-1517878091bd admin admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:42:06.250949 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a37a43-7450-4651-b49e-f4a8b1519c58 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbddf971a2354bbdbb9da1d63f3abf88', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0788e6725a84e57a4174284c1845755', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:42:06.251787 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a37a43-7450-4651-b49e-f4a8b1519c58 service cinder] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:42:06.252062 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a37a43-7450-4651-b49e-f4a8b1519c58 service cinder] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:42:06.252353 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a37a43-7450-4651-b49e-f4a8b1519c58 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:42:06.262959 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a37a43-7450-4651-b49e-f4a8b1519c58 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:42:06.264445 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a37a43-7450-4651-b49e-f4a8b1519c58 service cinder] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:42:06.287086 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1113/2206] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 1 16:42:06 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 01 16:42:06.301854 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5775729f-ec88-4312-9be5-12b1645c511f None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:42:06.303522 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-5775729f-ec88-4312-9be5-12b1645c511f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:42:06.417038 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5775729f-ec88-4312-9be5-12b1645c511f None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:42:06.421999 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a6296a-91e9-4748-8fab-8dcd6d3e27a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:42:06.422992 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a6296a-91e9-4748-8fab-8dcd6d3e27a1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:42:06.423282 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a6296a-91e9-4748-8fab-8dcd6d3e27a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:42:06.423566 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a6296a-91e9-4748-8fab-8dcd6d3e27a1 admin admin] PATH_INFO: `/v3/projects/523c0b53f60d435f8531e8434e7a8e11` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:42:06.424813 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a6296a-91e9-4748-8fab-8dcd6d3e27a1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=523c0b53f60d435f8531e8434e7a8e11)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:42:06.426813 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a6296a-91e9-4748-8fab-8dcd6d3e27a1 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:42:06.496425 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-52a6296a-91e9-4748-8fab-8dcd6d3e27a1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '523c0b53f60d435f8531e8434e7a8e11', 'request_id': 'req-52a6296a-91e9-4748-8fab-8dcd6d3e27a1'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:42:06.502557 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-52a6296a-91e9-4748-8fab-8dcd6d3e27a1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '523c0b53f60d435f8531e8434e7a8e11', 'request_id': 'req-52a6296a-91e9-4748-8fab-8dcd6d3e27a1'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:42:06.509706 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-52a6296a-91e9-4748-8fab-8dcd6d3e27a1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '523c0b53f60d435f8531e8434e7a8e11', 'request_id': 'req-52a6296a-91e9-4748-8fab-8dcd6d3e27a1'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:42:06.525300 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-52a6296a-91e9-4748-8fab-8dcd6d3e27a1 admin admin] The token cache is being invalidate because project 523c0b53f60d435f8531e8434e7a8e11 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80311) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 01 16:42:06.525503 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-52a6296a-91e9-4748-8fab-8dcd6d3e27a1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:42:06.526495 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-52a6296a-91e9-4748-8fab-8dcd6d3e27a1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:42:06.528141 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1094/2207] 10.222.0.57 () {58 vars in 1232 bytes} [Fri Jul 1 16:42:06 2022] DELETE /identity/v3/projects/523c0b53f60d435f8531e8434e7a8e11 => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 01 16:42:06.537881 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063a4103-77e1-4094-aaf2-303b68d1987f None None] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:42:06.537881 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063a4103-77e1-4094-aaf2-303b68d1987f None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:42:06.537881 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063a4103-77e1-4094-aaf2-303b68d1987f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:42:06.565867 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.auth.core [None req-063a4103-77e1-4094-aaf2-303b68d1987f None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80312) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:42:06.570531 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-063a4103-77e1-4094-aaf2-303b68d1987f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:42:06.624427 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:42:06.670589 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1114/2208] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 1 16:42:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 16:42:06.681715 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b104ff-fca9-4415-b09d-1f76c0c91018 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:42:06.682255 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b104ff-fca9-4415-b09d-1f76c0c91018 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:42:06.682656 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b104ff-fca9-4415-b09d-1f76c0c91018 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:42:06.728974 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-39b104ff-fca9-4415-b09d-1f76c0c91018 None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:42:06.730549 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-39b104ff-fca9-4415-b09d-1f76c0c91018 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:42:06.773359 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1095/2209] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 1 16:42:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 16:42:06.774206 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063a4103-77e1-4094-aaf2-303b68d1987f None None] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:42:06.791873 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063a4103-77e1-4094-aaf2-303b68d1987f None None] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:42:06.811463 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f879656-6be5-42c8-98bc-d80fd912f99f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:42:06.812301 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f879656-6be5-42c8-98bc-d80fd912f99f admin admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:42:06.813433 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f879656-6be5-42c8-98bc-d80fd912f99f admin admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:42:06.814002 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f879656-6be5-42c8-98bc-d80fd912f99f admin admin] PATH_INFO: `/v3/domains` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:42:06.814618 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f879656-6be5-42c8-98bc-d80fd912f99f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80312) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:42:06.815004 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f879656-6be5-42c8-98bc-d80fd912f99f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:42:06.816796 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f879656-6be5-42c8-98bc-d80fd912f99f admin admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:42:06.841935 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1115/2210] 10.222.0.57 () {58 vars in 1152 bytes} [Fri Jul 1 16:42:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 01 16:42:06.847836 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b104ff-fca9-4415-b09d-1f76c0c91018 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:42:06.861499 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b104ff-fca9-4415-b09d-1f76c0c91018 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:42:06.876108 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5590981-ecb8-4d68-99d6-bc3dbfae30c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:42:06.876795 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5590981-ecb8-4d68-99d6-bc3dbfae30c2 admin admin] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:42:06.879670 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5590981-ecb8-4d68-99d6-bc3dbfae30c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:42:06.880379 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5590981-ecb8-4d68-99d6-bc3dbfae30c2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:42:06.880692 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5590981-ecb8-4d68-99d6-bc3dbfae30c2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:42:06.884155 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5590981-ecb8-4d68-99d6-bc3dbfae30c2 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:42:06.920641 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f879656-6be5-42c8-98bc-d80fd912f99f admin admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:42:06.921580 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1096/2211] 10.222.0.57 () {60 vars in 1152 bytes} [Fri Jul 1 16:42:06 2022] POST /identity/v3/projects => generated 383 bytes in 74 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 16:42:06.927354 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-6f879656-6be5-42c8-98bc-d80fd912f99f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:42:06.935422 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5590981-ecb8-4d68-99d6-bc3dbfae30c2 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:42:06.952218 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:42:06.957038 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f879656-6be5-42c8-98bc-d80fd912f99f admin admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:42:07.108972 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037e27ca-4848-465a-98e8-5d104acf947f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:42:07.111077 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037e27ca-4848-465a-98e8-5d104acf947f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:42:07.111752 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037e27ca-4848-465a-98e8-5d104acf947f admin admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:42:07.112108 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037e27ca-4848-465a-98e8-5d104acf947f admin admin] PATH_INFO: `/v3/users/fc2248eef607473d84be9cb3e06c3f35` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:42:07.113606 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-037e27ca-4848-465a-98e8-5d104acf947f admin admin] RBAC: Authorizing `identity:delete_user(user_id=fc2248eef607473d84be9cb3e06c3f35)` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:42:07.116375 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-037e27ca-4848-465a-98e8-5d104acf947f admin admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:42:07.127268 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5590981-ecb8-4d68-99d6-bc3dbfae30c2 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:42:07.146285 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e49a848-def8-4524-bfef-dc6a87179973 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:42:07.147032 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e49a848-def8-4524-bfef-dc6a87179973 admin admin] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:42:07.147318 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e49a848-def8-4524-bfef-dc6a87179973 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:42:07.147595 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e49a848-def8-4524-bfef-dc6a87179973 admin admin] PATH_INFO: `/v3/users` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:42:07.148298 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e49a848-def8-4524-bfef-dc6a87179973 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:42:07.150261 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e49a848-def8-4524-bfef-dc6a87179973 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:42:07.206141 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1097/2212] 10.222.0.57 () {60 vars in 1143 bytes} [Fri Jul 1 16:42:06 2022] POST /identity/v3/users => generated 292 bytes in 268 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 16:42:07.210448 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-037e27ca-4848-465a-98e8-5d104acf947f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fc2248eef607473d84be9cb3e06c3f35'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:42:07.215543 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e49a848-def8-4524-bfef-dc6a87179973 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:42:07.238766 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e49a848-def8-4524-bfef-dc6a87179973 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:42:07.240943 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-037e27ca-4848-465a-98e8-5d104acf947f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fc2248eef607473d84be9cb3e06c3f35'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:42:07.246281 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008e904b-727c-484a-9b8f-cd79b8652f4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:42:07.246976 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008e904b-727c-484a-9b8f-cd79b8652f4a admin admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:42:07.247266 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008e904b-727c-484a-9b8f-cd79b8652f4a admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:42:07.247545 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008e904b-727c-484a-9b8f-cd79b8652f4a admin admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:42:07.248286 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-008e904b-727c-484a-9b8f-cd79b8652f4a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:42:07.249951 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-008e904b-727c-484a-9b8f-cd79b8652f4a admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:42:07.263716 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1098/2213] 10.222.0.57 () {58 vars in 1121 bytes} [Fri Jul 1 16:42:07 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 16:42:07.271647 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008e904b-727c-484a-9b8f-cd79b8652f4a admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:42:07.280562 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-037e27ca-4848-465a-98e8-5d104acf947f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fc2248eef607473d84be9cb3e06c3f35'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:42:07.282900 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-037e27ca-4848-465a-98e8-5d104acf947f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fc2248eef607473d84be9cb3e06c3f35'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:42:07.284428 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-037e27ca-4848-465a-98e8-5d104acf947f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fc2248eef607473d84be9cb3e06c3f35'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:42:07.290855 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008e904b-727c-484a-9b8f-cd79b8652f4a admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:42:07.296473 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11bcde8d-c303-4f36-9729-0bc0de740466 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:42:07.297735 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bcde8d-c303-4f36-9729-0bc0de740466 admin admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:42:07.298013 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bcde8d-c303-4f36-9729-0bc0de740466 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:42:07.298447 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bcde8d-c303-4f36-9729-0bc0de740466 admin admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:42:07.299285 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11bcde8d-c303-4f36-9729-0bc0de740466 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:42:07.321359 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11bcde8d-c303-4f36-9729-0bc0de740466 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:42:07.353210 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1116/2214] 10.222.0.57 () {58 vars in 1223 bytes} [Fri Jul 1 16:42:06 2022] DELETE /identity/v3/users/fc2248eef607473d84be9cb3e06c3f35 => generated 0 bytes in 433 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 16:42:07.357560 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037e27ca-4848-465a-98e8-5d104acf947f admin admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:42:07.360766 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1099/2215] 10.222.0.57 () {58 vars in 1121 bytes} [Fri Jul 1 16:42:07 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 16:42:07.361403 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-037e27ca-4848-465a-98e8-5d104acf947f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:42:07.362529 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ad4c90-de05-4822-a82b-43204774624b None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:42:07.362726 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ad4c90-de05-4822-a82b-43204774624b None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:42:07.363009 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ad4c90-de05-4822-a82b-43204774624b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:42:07.416719 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-e4ad4c90-de05-4822-a82b-43204774624b None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:42:07.418201 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-e4ad4c90-de05-4822-a82b-43204774624b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:42:07.471537 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:42:07.519036 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037e27ca-4848-465a-98e8-5d104acf947f admin admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:42:07.534803 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d0b9b2-1a59-4edf-9e1a-2c108f5552ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:42:07.537654 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d0b9b2-1a59-4edf-9e1a-2c108f5552ad admin admin] REQUEST_METHOD: `PUT` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:42:07.538160 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d0b9b2-1a59-4edf-9e1a-2c108f5552ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:42:07.538662 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d0b9b2-1a59-4edf-9e1a-2c108f5552ad admin admin] PATH_INFO: `/v3/projects/5aaa3579edcb415b97e290757ace1a11/users/69f8f9d699174915902141c963f74f85/roles/983be40bdab04c478eaa90c6b6a46b14` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:42:07.574897 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d0b9b2-1a59-4edf-9e1a-2c108f5552ad admin admin] RBAC: Authorizing `identity:create_grant(project_id=5aaa3579edcb415b97e290757ace1a11, user_id=69f8f9d699174915902141c963f74f85, role_id=983be40bdab04c478eaa90c6b6a46b14)` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:42:07.575901 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1100/2216] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 1 16:42:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 16:42:07.581458 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d0b9b2-1a59-4edf-9e1a-2c108f5552ad admin admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:42:07.587700 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ad4c90-de05-4822-a82b-43204774624b None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:42:07.589761 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-e4ad4c90-de05-4822-a82b-43204774624b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:42:07.614148 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1117/2217] 10.222.0.57 () {60 vars in 1482 bytes} [Fri Jul 1 16:42:07 2022] PUT /identity/v3/projects/5aaa3579edcb415b97e290757ace1a11/users/69f8f9d699174915902141c963f74f85/roles/983be40bdab04c478eaa90c6b6a46b14 => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 16:42:07.628742 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2363f2a-ec08-4d64-8000-322f36dfd08c None None] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:42:07.629212 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2363f2a-ec08-4d64-8000-322f36dfd08c None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:42:07.629566 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2363f2a-ec08-4d64-8000-322f36dfd08c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:42:07.689715 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.auth.core [None req-b2363f2a-ec08-4d64-8000-322f36dfd08c None None] MFA Rules not processed for user `69f8f9d699174915902141c963f74f85`. Rule list: `[]` (Enabled: `True`). {{(pid=80312) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:42:07.690993 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-b2363f2a-ec08-4d64-8000-322f36dfd08c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:42:07.714169 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ad4c90-de05-4822-a82b-43204774624b None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:42:07.743541 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:42:07.766085 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1118/2218] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 1 16:42:07 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 01 16:42:07.770636 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2363f2a-ec08-4d64-8000-322f36dfd08c None None] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:42:07.776741 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6db341-c4b9-4aa6-a6bf-9e154230abe2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ed200ce29684c5f80354fa9a8638fe2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0788e6725a84e57a4174284c1845755', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:42:07.777397 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6db341-c4b9-4aa6-a6bf-9e154230abe2 service neutron] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:42:07.777684 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6db341-c4b9-4aa6-a6bf-9e154230abe2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:42:07.777963 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6db341-c4b9-4aa6-a6bf-9e154230abe2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:42:07.849274 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2363f2a-ec08-4d64-8000-322f36dfd08c None None] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:42:07.850242 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed6db341-c4b9-4aa6-a6bf-9e154230abe2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:42:07.850611 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed6db341-c4b9-4aa6-a6bf-9e154230abe2 service neutron] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:42:07.855232 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8c0f60-cb5a-4509-abcc-8ab537ec5c95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:42:07.856022 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8c0f60-cb5a-4509-abcc-8ab537ec5c95 admin admin] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:42:07.856411 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8c0f60-cb5a-4509-abcc-8ab537ec5c95 admin admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:42:07.857204 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8c0f60-cb5a-4509-abcc-8ab537ec5c95 admin admin] PATH_INFO: `/v3/projects` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:42:07.857573 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec8c0f60-cb5a-4509-abcc-8ab537ec5c95 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:42:07.860559 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec8c0f60-cb5a-4509-abcc-8ab537ec5c95 admin admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:42:07.864348 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1101/2219] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 1 16:42:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 277 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 16:42:07.894387 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1119/2220] 10.222.0.57 () {60 vars in 1152 bytes} [Fri Jul 1 16:42:07 2022] POST /identity/v3/projects => generated 381 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 01 16:42:07.897338 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6db341-c4b9-4aa6-a6bf-9e154230abe2 service neutron] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:42:07.911256 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6db341-c4b9-4aa6-a6bf-9e154230abe2 service neutron] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:42:08.007253 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92cc1abc-21b5-400c-987c-167ce6bedeeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:42:08.007966 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cc1abc-21b5-400c-987c-167ce6bedeeb admin admin] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:42:08.008251 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cc1abc-21b5-400c-987c-167ce6bedeeb admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:42:08.008530 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cc1abc-21b5-400c-987c-167ce6bedeeb admin admin] PATH_INFO: `/v3/users` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:42:08.009535 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92cc1abc-21b5-400c-987c-167ce6bedeeb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:42:08.012175 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92cc1abc-21b5-400c-987c-167ce6bedeeb admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:42:08.036614 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1102/2221] 10.222.0.57 () {60 vars in 1143 bytes} [Fri Jul 1 16:42:07 2022] POST /identity/v3/users => generated 291 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 16:42:08.040318 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8c0f60-cb5a-4509-abcc-8ab537ec5c95 admin admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:42:08.057330 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8c0f60-cb5a-4509-abcc-8ab537ec5c95 admin admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:42:08.065316 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e83ae7d-15f0-4ae7-bca2-2a12dfdf2d7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:42:08.066662 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e83ae7d-15f0-4ae7-bca2-2a12dfdf2d7e admin admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:42:08.067196 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e83ae7d-15f0-4ae7-bca2-2a12dfdf2d7e admin admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:42:08.067687 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e83ae7d-15f0-4ae7-bca2-2a12dfdf2d7e admin admin] PATH_INFO: `/v3/roles` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:42:08.068709 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e83ae7d-15f0-4ae7-bca2-2a12dfdf2d7e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:42:08.071387 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e83ae7d-15f0-4ae7-bca2-2a12dfdf2d7e admin admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:42:08.088839 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1120/2222] 10.222.0.57 () {58 vars in 1121 bytes} [Fri Jul 1 16:42:08 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 16:42:08.092731 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92cc1abc-21b5-400c-987c-167ce6bedeeb admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:42:08.104461 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92cc1abc-21b5-400c-987c-167ce6bedeeb admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:42:08.110254 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717f4a64-eaf4-46f9-b100-80a973f5e0a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:42:08.111111 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717f4a64-eaf4-46f9-b100-80a973f5e0a9 admin admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:42:08.111560 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717f4a64-eaf4-46f9-b100-80a973f5e0a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:42:08.111889 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717f4a64-eaf4-46f9-b100-80a973f5e0a9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:42:08.112689 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-717f4a64-eaf4-46f9-b100-80a973f5e0a9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:42:08.115402 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-717f4a64-eaf4-46f9-b100-80a973f5e0a9 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:42:08.137611 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1103/2223] 10.222.0.57 () {58 vars in 1121 bytes} [Fri Jul 1 16:42:08 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 16:42:08.142049 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e83ae7d-15f0-4ae7-bca2-2a12dfdf2d7e admin admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:42:08.158568 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e83ae7d-15f0-4ae7-bca2-2a12dfdf2d7e admin admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:42:08.165047 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270539d2-380e-47d9-bb5e-63f0cacd0928 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:42:08.166097 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270539d2-380e-47d9-bb5e-63f0cacd0928 admin admin] REQUEST_METHOD: `PUT` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:42:08.166409 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270539d2-380e-47d9-bb5e-63f0cacd0928 admin admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:42:08.166687 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270539d2-380e-47d9-bb5e-63f0cacd0928 admin admin] PATH_INFO: `/v3/projects/2553a9f2d1b24630be799acce7a93f84/users/79c7caef9d964b9088aa62d12f83224e/roles/983be40bdab04c478eaa90c6b6a46b14` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:42:08.184211 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-270539d2-380e-47d9-bb5e-63f0cacd0928 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2553a9f2d1b24630be799acce7a93f84, user_id=79c7caef9d964b9088aa62d12f83224e, role_id=983be40bdab04c478eaa90c6b6a46b14)` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:42:08.187030 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-270539d2-380e-47d9-bb5e-63f0cacd0928 admin admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:42:08.204789 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1121/2224] 10.222.0.57 () {60 vars in 1482 bytes} [Fri Jul 1 16:42:08 2022] PUT /identity/v3/projects/2553a9f2d1b24630be799acce7a93f84/users/79c7caef9d964b9088aa62d12f83224e/roles/983be40bdab04c478eaa90c6b6a46b14 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 16:42:08.217467 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118ceb9b-7f84-4d2e-907b-15774512338d None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:42:08.218025 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118ceb9b-7f84-4d2e-907b-15774512338d None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:42:08.218219 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118ceb9b-7f84-4d2e-907b-15774512338d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:42:08.248734 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-118ceb9b-7f84-4d2e-907b-15774512338d None None] MFA Rules not processed for user `79c7caef9d964b9088aa62d12f83224e`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:42:08.250750 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-118ceb9b-7f84-4d2e-907b-15774512338d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:42:08.314993 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:42:08.340321 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1104/2225] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 1 16:42:08 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 16:42:08.345855 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270539d2-380e-47d9-bb5e-63f0cacd0928 admin admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:42:08.363590 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:42:08.367498 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118ceb9b-7f84-4d2e-907b-15774512338d None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:42:08.370397 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-118ceb9b-7f84-4d2e-907b-15774512338d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:42:08.447814 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270539d2-380e-47d9-bb5e-63f0cacd0928 admin admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:42:08.458085 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fab1fb1-c4bc-42d1-a25f-e40aa59cd741 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:42:08.459052 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fab1fb1-c4bc-42d1-a25f-e40aa59cd741 admin admin] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:42:08.459452 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fab1fb1-c4bc-42d1-a25f-e40aa59cd741 admin admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:42:08.460073 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fab1fb1-c4bc-42d1-a25f-e40aa59cd741 admin admin] PATH_INFO: `/v3/projects` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:42:08.460894 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fab1fb1-c4bc-42d1-a25f-e40aa59cd741 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:42:08.468271 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fab1fb1-c4bc-42d1-a25f-e40aa59cd741 admin admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:42:08.476056 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118ceb9b-7f84-4d2e-907b-15774512338d None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:42:08.486048 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966e4ec6-f0c7-4882-a908-9b72e5824ed9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:42:08.486963 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966e4ec6-f0c7-4882-a908-9b72e5824ed9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:42:08.487250 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966e4ec6-f0c7-4882-a908-9b72e5824ed9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:42:08.487532 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966e4ec6-f0c7-4882-a908-9b72e5824ed9 admin admin] PATH_INFO: `/v3/projects/46b020c27df14f6885dd3daf5271c2f0` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:42:08.489353 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-966e4ec6-f0c7-4882-a908-9b72e5824ed9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=46b020c27df14f6885dd3daf5271c2f0)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:42:08.491706 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-966e4ec6-f0c7-4882-a908-9b72e5824ed9 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:42:08.511546 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1122/2226] 10.222.0.57 () {60 vars in 1152 bytes} [Fri Jul 1 16:42:08 2022] POST /identity/v3/projects => generated 383 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 16:42:08.512801 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fab1fb1-c4bc-42d1-a25f-e40aa59cd741 admin admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:42:08.555067 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-966e4ec6-f0c7-4882-a908-9b72e5824ed9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '46b020c27df14f6885dd3daf5271c2f0', 'request_id': 'req-966e4ec6-f0c7-4882-a908-9b72e5824ed9'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:42:08.566753 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-966e4ec6-f0c7-4882-a908-9b72e5824ed9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '46b020c27df14f6885dd3daf5271c2f0', 'request_id': 'req-966e4ec6-f0c7-4882-a908-9b72e5824ed9'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:42:08.573875 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-966e4ec6-f0c7-4882-a908-9b72e5824ed9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '46b020c27df14f6885dd3daf5271c2f0', 'request_id': 'req-966e4ec6-f0c7-4882-a908-9b72e5824ed9'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:42:08.592986 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fab1fb1-c4bc-42d1-a25f-e40aa59cd741 admin admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:42:08.597067 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-966e4ec6-f0c7-4882-a908-9b72e5824ed9 admin admin] The token cache is being invalidate because project 46b020c27df14f6885dd3daf5271c2f0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80311) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 01 16:42:08.597358 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-966e4ec6-f0c7-4882-a908-9b72e5824ed9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:42:08.598137 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-966e4ec6-f0c7-4882-a908-9b72e5824ed9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:42:08.602385 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1105/2227] 10.222.0.57 () {58 vars in 1232 bytes} [Fri Jul 1 16:42:08 2022] DELETE /identity/v3/projects/46b020c27df14f6885dd3daf5271c2f0 => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 16:42:08.604693 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966e4ec6-f0c7-4882-a908-9b72e5824ed9 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:42:08.606383 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-966e4ec6-f0c7-4882-a908-9b72e5824ed9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:42:08.655263 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7d9a71-5563-4e5a-9f18-7ede91f2b18d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:42:08.655978 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7d9a71-5563-4e5a-9f18-7ede91f2b18d admin admin] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:42:08.656348 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7d9a71-5563-4e5a-9f18-7ede91f2b18d admin admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:42:08.656795 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7d9a71-5563-4e5a-9f18-7ede91f2b18d admin admin] PATH_INFO: `/v3/users` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:42:08.657490 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd7d9a71-5563-4e5a-9f18-7ede91f2b18d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:42:08.659438 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd7d9a71-5563-4e5a-9f18-7ede91f2b18d admin admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:42:08.675001 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966e4ec6-f0c7-4882-a908-9b72e5824ed9 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:42:08.679970 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db877662-3874-4b71-adb2-b4d9f7fdca49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:42:08.683425 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1123/2228] 10.222.0.57 () {60 vars in 1143 bytes} [Fri Jul 1 16:42:08 2022] POST /identity/v3/users => generated 298 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 16:42:08.683583 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db877662-3874-4b71-adb2-b4d9f7fdca49 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:42:08.683583 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db877662-3874-4b71-adb2-b4d9f7fdca49 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:42:08.683583 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db877662-3874-4b71-adb2-b4d9f7fdca49 admin admin] PATH_INFO: `/v3/users/17a8ba7ee4ab4a348bf000cd654f5c89` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:42:08.683583 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db877662-3874-4b71-adb2-b4d9f7fdca49 admin admin] RBAC: Authorizing `identity:delete_user(user_id=17a8ba7ee4ab4a348bf000cd654f5c89)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:42:08.684374 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7d9a71-5563-4e5a-9f18-7ede91f2b18d admin admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:42:08.684496 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db877662-3874-4b71-adb2-b4d9f7fdca49 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:42:08.686017 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-cd7d9a71-5563-4e5a-9f18-7ede91f2b18d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:42:08.698690 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7d9a71-5563-4e5a-9f18-7ede91f2b18d admin admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:42:08.704465 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06cf3cf3-c680-4c85-af2c-e1ddd412bcbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:42:08.705851 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:42:08.706755 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cf3cf3-c680-4c85-af2c-e1ddd412bcbb admin admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:42:08.706917 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cf3cf3-c680-4c85-af2c-e1ddd412bcbb admin admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:42:08.707157 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cf3cf3-c680-4c85-af2c-e1ddd412bcbb admin admin] PATH_INFO: `/v3/roles` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:42:08.708113 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06cf3cf3-c680-4c85-af2c-e1ddd412bcbb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:42:08.710091 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06cf3cf3-c680-4c85-af2c-e1ddd412bcbb admin admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:42:08.724722 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1124/2229] 10.222.0.57 () {58 vars in 1121 bytes} [Fri Jul 1 16:42:08 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 16:42:08.728990 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06cf3cf3-c680-4c85-af2c-e1ddd412bcbb admin admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:42:08.743656 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06cf3cf3-c680-4c85-af2c-e1ddd412bcbb admin admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:42:08.746377 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-db877662-3874-4b71-adb2-b4d9f7fdca49 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '17a8ba7ee4ab4a348bf000cd654f5c89'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:42:08.746377 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-db877662-3874-4b71-adb2-b4d9f7fdca49 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '17a8ba7ee4ab4a348bf000cd654f5c89'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:42:08.746377 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-db877662-3874-4b71-adb2-b4d9f7fdca49 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '17a8ba7ee4ab4a348bf000cd654f5c89'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:42:08.749171 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64b209a-d7af-4969-a8b5-5876bb908f54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:42:08.750976 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64b209a-d7af-4969-a8b5-5876bb908f54 admin admin] REQUEST_METHOD: `PUT` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:42:08.751400 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64b209a-d7af-4969-a8b5-5876bb908f54 admin admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:42:08.751549 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64b209a-d7af-4969-a8b5-5876bb908f54 admin admin] PATH_INFO: `/v3/projects/bed5cb2d123347d2ae9173d6ae91fbc2/users/9024df4e5a7241fea4d1a3040c56dc46/roles/84ee739146934c1ab32c501ff8093644` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:42:08.755777 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-db877662-3874-4b71-adb2-b4d9f7fdca49 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '17a8ba7ee4ab4a348bf000cd654f5c89'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:42:08.771024 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a64b209a-d7af-4969-a8b5-5876bb908f54 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bed5cb2d123347d2ae9173d6ae91fbc2, user_id=9024df4e5a7241fea4d1a3040c56dc46, role_id=84ee739146934c1ab32c501ff8093644)` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:42:08.772384 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-db877662-3874-4b71-adb2-b4d9f7fdca49 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '17a8ba7ee4ab4a348bf000cd654f5c89'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:42:08.773739 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a64b209a-d7af-4969-a8b5-5876bb908f54 admin admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:42:08.787680 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:42:08.794987 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1125/2230] 10.222.0.57 () {60 vars in 1482 bytes} [Fri Jul 1 16:42:08 2022] PUT /identity/v3/projects/bed5cb2d123347d2ae9173d6ae91fbc2/users/9024df4e5a7241fea4d1a3040c56dc46/roles/84ee739146934c1ab32c501ff8093644 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 16:42:08.798412 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1106/2231] 10.222.0.57 () {58 vars in 1223 bytes} [Fri Jul 1 16:42:08 2022] DELETE /identity/v3/users/17a8ba7ee4ab4a348bf000cd654f5c89 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 01 16:42:08.800564 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64b209a-d7af-4969-a8b5-5876bb908f54 admin admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:42:08.802574 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-a64b209a-d7af-4969-a8b5-5876bb908f54 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:42:08.896058 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64b209a-d7af-4969-a8b5-5876bb908f54 admin admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:42:08.956972 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86904202-29aa-4920-b430-2d3ec2ea7ee2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:42:08.957717 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86904202-29aa-4920-b430-2d3ec2ea7ee2 admin admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:42:08.957929 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86904202-29aa-4920-b430-2d3ec2ea7ee2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:42:08.958214 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86904202-29aa-4920-b430-2d3ec2ea7ee2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:42:08.959064 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86904202-29aa-4920-b430-2d3ec2ea7ee2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:42:08.961338 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86904202-29aa-4920-b430-2d3ec2ea7ee2 admin admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:42:08.977580 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1126/2232] 10.222.0.57 () {58 vars in 1121 bytes} [Fri Jul 1 16:42:08 2022] GET /identity/v3/roles => generated 1420 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 16:42:08.978289 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db877662-3874-4b71-adb2-b4d9f7fdca49 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:42:08.991566 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db877662-3874-4b71-adb2-b4d9f7fdca49 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:42:08.997729 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d39268c-2671-4a65-9865-cfe0156f6080 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:42:08.999134 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d39268c-2671-4a65-9865-cfe0156f6080 admin admin] REQUEST_METHOD: `PUT` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:42:08.999422 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d39268c-2671-4a65-9865-cfe0156f6080 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:42:08.999726 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d39268c-2671-4a65-9865-cfe0156f6080 admin admin] PATH_INFO: `/v3/projects/bed5cb2d123347d2ae9173d6ae91fbc2/users/9024df4e5a7241fea4d1a3040c56dc46/roles/84ee739146934c1ab32c501ff8093644` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:42:09.007966 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d39268c-2671-4a65-9865-cfe0156f6080 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bed5cb2d123347d2ae9173d6ae91fbc2, user_id=9024df4e5a7241fea4d1a3040c56dc46, role_id=84ee739146934c1ab32c501ff8093644)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:42:09.011108 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d39268c-2671-4a65-9865-cfe0156f6080 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:42:09.026930 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1107/2233] 10.222.0.57 () {60 vars in 1482 bytes} [Fri Jul 1 16:42:08 2022] PUT /identity/v3/projects/bed5cb2d123347d2ae9173d6ae91fbc2/users/9024df4e5a7241fea4d1a3040c56dc46/roles/84ee739146934c1ab32c501ff8093644 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 01 16:42:09.037972 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2056e0b4-ea37-4c4d-b8d2-9c13926d6c98 None None] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:42:09.038526 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2056e0b4-ea37-4c4d-b8d2-9c13926d6c98 None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:42:09.039081 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2056e0b4-ea37-4c4d-b8d2-9c13926d6c98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:42:09.094666 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.auth.core [None req-2056e0b4-ea37-4c4d-b8d2-9c13926d6c98 None None] MFA Rules not processed for user `9024df4e5a7241fea4d1a3040c56dc46`. Rule list: `[]` (Enabled: `True`). {{(pid=80312) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:42:09.096195 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-2056e0b4-ea37-4c4d-b8d2-9c13926d6c98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:42:09.113613 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d39268c-2671-4a65-9865-cfe0156f6080 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:42:09.115475 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-4d39268c-2671-4a65-9865-cfe0156f6080 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:42:09.204199 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1127/2234] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 1 16:42:09 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 16:42:09.222524 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2056e0b4-ea37-4c4d-b8d2-9c13926d6c98 None None] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:42:09.223953 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-2056e0b4-ea37-4c4d-b8d2-9c13926d6c98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:42:09.238126 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d39268c-2671-4a65-9865-cfe0156f6080 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:42:09.245495 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f9ab2f-bf8f-4965-b39a-5f2fd3c22894 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:42:09.246732 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f9ab2f-bf8f-4965-b39a-5f2fd3c22894 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:42:09.247175 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f9ab2f-bf8f-4965-b39a-5f2fd3c22894 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:42:09.248230 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f9ab2f-bf8f-4965-b39a-5f2fd3c22894 admin admin] PATH_INFO: `/v3/projects/f381929ffd944e5f8d19d9beaed7d9bf` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:42:09.250396 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f9ab2f-bf8f-4965-b39a-5f2fd3c22894 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f381929ffd944e5f8d19d9beaed7d9bf)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:42:09.254204 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f9ab2f-bf8f-4965-b39a-5f2fd3c22894 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:42:09.311107 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=100 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:42:09.350925 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2056e0b4-ea37-4c4d-b8d2-9c13926d6c98 None None] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:42:09.374198 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-06f9ab2f-bf8f-4965-b39a-5f2fd3c22894 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f381929ffd944e5f8d19d9beaed7d9bf', 'request_id': 'req-06f9ab2f-bf8f-4965-b39a-5f2fd3c22894'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:42:09.394982 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-06f9ab2f-bf8f-4965-b39a-5f2fd3c22894 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f381929ffd944e5f8d19d9beaed7d9bf', 'request_id': 'req-06f9ab2f-bf8f-4965-b39a-5f2fd3c22894'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:42:09.404947 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-06f9ab2f-bf8f-4965-b39a-5f2fd3c22894 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f381929ffd944e5f8d19d9beaed7d9bf', 'request_id': 'req-06f9ab2f-bf8f-4965-b39a-5f2fd3c22894'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:42:09.435877 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-06f9ab2f-bf8f-4965-b39a-5f2fd3c22894 admin admin] The token cache is being invalidate because project f381929ffd944e5f8d19d9beaed7d9bf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80311) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 01 16:42:09.436321 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-06f9ab2f-bf8f-4965-b39a-5f2fd3c22894 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:42:09.439050 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-06f9ab2f-bf8f-4965-b39a-5f2fd3c22894 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:42:09.441126 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1108/2235] 10.222.0.57 () {58 vars in 1232 bytes} [Fri Jul 1 16:42:09 2022] DELETE /identity/v3/projects/f381929ffd944e5f8d19d9beaed7d9bf => generated 0 bytes in 328 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 16:42:09.446559 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41ebf0e-fe06-480e-8477-4b48764d26e8 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:42:09.446559 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41ebf0e-fe06-480e-8477-4b48764d26e8 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:42:09.446905 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41ebf0e-fe06-480e-8477-4b48764d26e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:42:09.470715 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965ef070-e732-47b1-941f-7002b8f15098 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbddf971a2354bbdbb9da1d63f3abf88', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0788e6725a84e57a4174284c1845755', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:42:09.470881 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965ef070-e732-47b1-941f-7002b8f15098 service cinder] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:42:09.470963 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-b41ebf0e-fe06-480e-8477-4b48764d26e8 None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:42:09.471048 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965ef070-e732-47b1-941f-7002b8f15098 service cinder] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:42:09.472084 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965ef070-e732-47b1-941f-7002b8f15098 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:42:09.472199 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-b41ebf0e-fe06-480e-8477-4b48764d26e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:42:09.473523 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-965ef070-e732-47b1-941f-7002b8f15098 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:42:09.478954 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:DELETE=16,keystone:SELECT=81,keystone:INSERT=3 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:42:09.644223 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1109/2236] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 1 16:42:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 16:42:09.647811 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-965ef070-e732-47b1-941f-7002b8f15098 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:42:09.651497 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-965ef070-e732-47b1-941f-7002b8f15098 service cinder] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:42:09.653692 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6081c795-dba5-47c3-a802-e04d5a5dcc19 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:42:09.653990 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6081c795-dba5-47c3-a802-e04d5a5dcc19 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:42:09.654279 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6081c795-dba5-47c3-a802-e04d5a5dcc19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:42:09.667427 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1128/2237] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 1 16:42:09 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 444 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 01 16:42:09.718169 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-6081c795-dba5-47c3-a802-e04d5a5dcc19 None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:42:09.719826 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-6081c795-dba5-47c3-a802-e04d5a5dcc19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:42:09.758026 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1110/2238] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 1 16:42:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 01 16:42:09.763692 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965ef070-e732-47b1-941f-7002b8f15098 service cinder] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:42:09.793829 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965ef070-e732-47b1-941f-7002b8f15098 service cinder] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:42:09.814862 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146c9653-3e4b-458c-95eb-d20fa7b221a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:42:09.817028 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146c9653-3e4b-458c-95eb-d20fa7b221a5 admin admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:42:09.817746 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146c9653-3e4b-458c-95eb-d20fa7b221a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:42:09.819254 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146c9653-3e4b-458c-95eb-d20fa7b221a5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:42:09.820663 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-146c9653-3e4b-458c-95eb-d20fa7b221a5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80312) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:42:09.821691 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-146c9653-3e4b-458c-95eb-d20fa7b221a5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:42:09.827028 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-146c9653-3e4b-458c-95eb-d20fa7b221a5 admin admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:42:09.888055 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1129/2239] 10.222.0.57 () {58 vars in 1152 bytes} [Fri Jul 1 16:42:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 126 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:42:09.889063 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6081c795-dba5-47c3-a802-e04d5a5dcc19 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:42:09.905861 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6081c795-dba5-47c3-a802-e04d5a5dcc19 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:42:09.919273 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062d3b28-de74-46b1-88bc-380deeee07d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:42:09.925967 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062d3b28-de74-46b1-88bc-380deeee07d3 admin admin] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:42:09.926279 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062d3b28-de74-46b1-88bc-380deeee07d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:42:09.926593 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062d3b28-de74-46b1-88bc-380deeee07d3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:42:09.929490 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-062d3b28-de74-46b1-88bc-380deeee07d3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:42:09.932842 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-062d3b28-de74-46b1-88bc-380deeee07d3 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:42:09.996287 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1111/2240] 10.222.0.57 () {60 vars in 1152 bytes} [Fri Jul 1 16:42:09 2022] POST /identity/v3/projects => generated 377 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 16:42:10.001004 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146c9653-3e4b-458c-95eb-d20fa7b221a5 admin admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:42:10.024088 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146c9653-3e4b-458c-95eb-d20fa7b221a5 admin admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:42:10.241888 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc6bc64-acc8-421d-8945-c8d7b7a95cfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:42:10.242618 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc6bc64-acc8-421d-8945-c8d7b7a95cfe admin admin] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:42:10.242908 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc6bc64-acc8-421d-8945-c8d7b7a95cfe admin admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:42:10.243189 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc6bc64-acc8-421d-8945-c8d7b7a95cfe admin admin] PATH_INFO: `/v3/users` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:42:10.243896 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc6bc64-acc8-421d-8945-c8d7b7a95cfe admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:42:10.246022 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc6bc64-acc8-421d-8945-c8d7b7a95cfe admin admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:42:10.300627 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1130/2241] 10.222.0.57 () {60 vars in 1143 bytes} [Fri Jul 1 16:42:09 2022] POST /identity/v3/users => generated 289 bytes in 301 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 16:42:10.301182 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062d3b28-de74-46b1-88bc-380deeee07d3 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:42:10.316600 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062d3b28-de74-46b1-88bc-380deeee07d3 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:42:10.336017 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4170156-b811-4b5b-9bb8-97da05b2d8aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:42:10.337452 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4170156-b811-4b5b-9bb8-97da05b2d8aa admin admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:42:10.339134 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4170156-b811-4b5b-9bb8-97da05b2d8aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:42:10.340983 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4170156-b811-4b5b-9bb8-97da05b2d8aa admin admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:42:10.344044 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4170156-b811-4b5b-9bb8-97da05b2d8aa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:42:10.348755 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4170156-b811-4b5b-9bb8-97da05b2d8aa admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:42:10.380490 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1112/2242] 10.222.0.57 () {58 vars in 1121 bytes} [Fri Jul 1 16:42:10 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 16:42:10.384772 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc6bc64-acc8-421d-8945-c8d7b7a95cfe admin admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:42:10.405267 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc6bc64-acc8-421d-8945-c8d7b7a95cfe admin admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:42:10.410960 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0edd44-aba6-46c7-b306-65303b58d4fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:42:10.411946 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0edd44-aba6-46c7-b306-65303b58d4fd admin admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:42:10.411946 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0edd44-aba6-46c7-b306-65303b58d4fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:42:10.412249 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0edd44-aba6-46c7-b306-65303b58d4fd admin admin] PATH_INFO: `/v3/roles` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:42:10.413016 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb0edd44-aba6-46c7-b306-65303b58d4fd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:42:10.414783 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb0edd44-aba6-46c7-b306-65303b58d4fd admin admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:42:10.427976 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1131/2243] 10.222.0.57 () {58 vars in 1121 bytes} [Fri Jul 1 16:42:10 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 16:42:10.432226 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4170156-b811-4b5b-9bb8-97da05b2d8aa admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:42:10.449342 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4170156-b811-4b5b-9bb8-97da05b2d8aa admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:42:10.454988 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463987c5-16be-4bcc-93f1-49314f34d2e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:42:10.456016 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463987c5-16be-4bcc-93f1-49314f34d2e1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:42:10.456373 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463987c5-16be-4bcc-93f1-49314f34d2e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:42:10.456857 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463987c5-16be-4bcc-93f1-49314f34d2e1 admin admin] PATH_INFO: `/v3/projects/c30c284bffb441f0b177ffd254af9c5a/users/414722f07c4c484a89f9ac5e80c7bab8/roles/983be40bdab04c478eaa90c6b6a46b14` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:42:10.480471 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463987c5-16be-4bcc-93f1-49314f34d2e1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c30c284bffb441f0b177ffd254af9c5a, user_id=414722f07c4c484a89f9ac5e80c7bab8, role_id=983be40bdab04c478eaa90c6b6a46b14)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:42:10.483476 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463987c5-16be-4bcc-93f1-49314f34d2e1 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:42:10.502565 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1113/2244] 10.222.0.57 () {60 vars in 1482 bytes} [Fri Jul 1 16:42:10 2022] PUT /identity/v3/projects/c30c284bffb441f0b177ffd254af9c5a/users/414722f07c4c484a89f9ac5e80c7bab8/roles/983be40bdab04c478eaa90c6b6a46b14 => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 01 16:42:10.513842 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d505ff9b-3e24-4643-8540-26af29945845 None None] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:42:10.514019 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d505ff9b-3e24-4643-8540-26af29945845 None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:42:10.514850 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d505ff9b-3e24-4643-8540-26af29945845 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:42:10.529697 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:42:10.565396 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.auth.core [None req-d505ff9b-3e24-4643-8540-26af29945845 None None] MFA Rules not processed for user `414722f07c4c484a89f9ac5e80c7bab8`. Rule list: `[]` (Enabled: `True`). {{(pid=80312) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:42:10.566935 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-d505ff9b-3e24-4643-8540-26af29945845 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:42:10.648542 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1132/2245] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 1 16:42:10 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 16:42:10.664691 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463987c5-16be-4bcc-93f1-49314f34d2e1 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:42:10.666576 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-463987c5-16be-4bcc-93f1-49314f34d2e1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:42:10.769352 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463987c5-16be-4bcc-93f1-49314f34d2e1 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:42:10.774474 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c78393-9c95-45fd-a6fe-63e9ba9f0d5b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbddf971a2354bbdbb9da1d63f3abf88', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0788e6725a84e57a4174284c1845755', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:42:10.775351 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c78393-9c95-45fd-a6fe-63e9ba9f0d5b service cinder] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:42:10.775636 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c78393-9c95-45fd-a6fe-63e9ba9f0d5b service cinder] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:42:10.775915 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c78393-9c95-45fd-a6fe-63e9ba9f0d5b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:42:10.786239 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c78393-9c95-45fd-a6fe-63e9ba9f0d5b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:42:10.787723 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c78393-9c95-45fd-a6fe-63e9ba9f0d5b service cinder] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:42:10.799600 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1114/2246] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 1 16:42:10 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 16:42:11.437740 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d505ff9b-3e24-4643-8540-26af29945845 None None] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:42:11.439272 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-d505ff9b-3e24-4643-8540-26af29945845 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:42:11.450966 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d505ff9b-3e24-4643-8540-26af29945845 None None] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:42:11.455074 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1840edc-ce06-4765-969b-7bb552a3c0d9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbddf971a2354bbdbb9da1d63f3abf88', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0788e6725a84e57a4174284c1845755', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:42:11.455729 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1840edc-ce06-4765-969b-7bb552a3c0d9 service cinder] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:42:11.456070 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1840edc-ce06-4765-969b-7bb552a3c0d9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:42:11.456288 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1840edc-ce06-4765-969b-7bb552a3c0d9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:42:11.458216 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-d1840edc-ce06-4765-969b-7bb552a3c0d9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:42:11.521720 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1840edc-ce06-4765-969b-7bb552a3c0d9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:42:11.523208 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1840edc-ce06-4765-969b-7bb552a3c0d9 service cinder] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:42:11.532765 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1133/2247] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 1 16:42:11 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 16:42:12.088693 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c78393-9c95-45fd-a6fe-63e9ba9f0d5b service cinder] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:42:12.097483 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c78393-9c95-45fd-a6fe-63e9ba9f0d5b service cinder] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:42:12.101223 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24949cc9-3fed-4dfd-ba44-9e53a1a093d4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbddf971a2354bbdbb9da1d63f3abf88', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0788e6725a84e57a4174284c1845755', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:42:12.101858 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24949cc9-3fed-4dfd-ba44-9e53a1a093d4 service cinder] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:42:12.102170 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24949cc9-3fed-4dfd-ba44-9e53a1a093d4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:42:12.102476 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24949cc9-3fed-4dfd-ba44-9e53a1a093d4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:42:12.104227 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-24949cc9-3fed-4dfd-ba44-9e53a1a093d4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:42:12.153083 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:42:12.168948 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24949cc9-3fed-4dfd-ba44-9e53a1a093d4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:42:12.170396 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24949cc9-3fed-4dfd-ba44-9e53a1a093d4 service cinder] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:42:12.180664 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1115/2248] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 1 16:42:12 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 16:42:18.651305 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db07085c-de19-4956-9ae8-51aab43c8109 None None] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:42:18.651804 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db07085c-de19-4956-9ae8-51aab43c8109 None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:42:18.651907 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db07085c-de19-4956-9ae8-51aab43c8109 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:42:18.670401 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.auth.core [None req-db07085c-de19-4956-9ae8-51aab43c8109 None None] MFA Rules not processed for user `7c4da21fbfa3472cac6e766a2f4799d8`. Rule list: `[]` (Enabled: `True`). {{(pid=80312) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:42:18.672543 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-db07085c-de19-4956-9ae8-51aab43c8109 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:42:18.737988 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1134/2249] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 1 16:42:18 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 16:42:20.367329 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24949cc9-3fed-4dfd-ba44-9e53a1a093d4 service cinder] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:42:20.368824 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-24949cc9-3fed-4dfd-ba44-9e53a1a093d4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:42:20.446785 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24949cc9-3fed-4dfd-ba44-9e53a1a093d4 service cinder] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:42:20.451797 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741754ec-53a1-45cf-a48a-e45491912f1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:42:20.453702 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741754ec-53a1-45cf-a48a-e45491912f1f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:42:20.454016 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741754ec-53a1-45cf-a48a-e45491912f1f admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:42:20.454505 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741754ec-53a1-45cf-a48a-e45491912f1f admin admin] PATH_INFO: `/v3/users/69f8f9d699174915902141c963f74f85` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:42:20.455951 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-741754ec-53a1-45cf-a48a-e45491912f1f admin admin] RBAC: Authorizing `identity:delete_user(user_id=69f8f9d699174915902141c963f74f85)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:42:20.457948 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-741754ec-53a1-45cf-a48a-e45491912f1f admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:42:20.521727 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-741754ec-53a1-45cf-a48a-e45491912f1f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '69f8f9d699174915902141c963f74f85'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:42:20.523045 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-741754ec-53a1-45cf-a48a-e45491912f1f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '69f8f9d699174915902141c963f74f85'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:42:20.523694 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-741754ec-53a1-45cf-a48a-e45491912f1f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '69f8f9d699174915902141c963f74f85'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:42:20.532710 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-741754ec-53a1-45cf-a48a-e45491912f1f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '69f8f9d699174915902141c963f74f85'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:42:20.548578 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-741754ec-53a1-45cf-a48a-e45491912f1f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '69f8f9d699174915902141c963f74f85'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:42:20.562380 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1116/2250] 10.222.0.57 () {58 vars in 1223 bytes} [Fri Jul 1 16:42:20 2022] DELETE /identity/v3/users/69f8f9d699174915902141c963f74f85 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 16:42:20.567181 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5e2d1b-2247-4787-8384-18cc594cbbd0 None None] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:42:20.567640 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5e2d1b-2247-4787-8384-18cc594cbbd0 None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:42:20.568039 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5e2d1b-2247-4787-8384-18cc594cbbd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:42:20.586908 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.auth.core [None req-7e5e2d1b-2247-4787-8384-18cc594cbbd0 None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80312) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:42:20.588205 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-7e5e2d1b-2247-4787-8384-18cc594cbbd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:42:20.626260 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=100 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:42:20.668984 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1135/2251] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 1 16:42:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 16:42:20.679368 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741754ec-53a1-45cf-a48a-e45491912f1f admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:42:20.680995 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-741754ec-53a1-45cf-a48a-e45491912f1f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:42:20.748095 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741754ec-53a1-45cf-a48a-e45491912f1f admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:42:20.754544 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ae0c4a-c733-4f90-b34d-3437464db19b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ed200ce29684c5f80354fa9a8638fe2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0788e6725a84e57a4174284c1845755', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:42:20.755206 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ae0c4a-c733-4f90-b34d-3437464db19b service neutron] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:42:20.755502 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ae0c4a-c733-4f90-b34d-3437464db19b service neutron] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:42:20.755774 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ae0c4a-c733-4f90-b34d-3437464db19b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:42:20.765935 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ae0c4a-c733-4f90-b34d-3437464db19b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:42:20.767410 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ae0c4a-c733-4f90-b34d-3437464db19b service neutron] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:42:20.777545 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1117/2252] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 1 16:42:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 16:42:21.069696 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5e2d1b-2247-4787-8384-18cc594cbbd0 None None] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:42:21.071634 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-7e5e2d1b-2247-4787-8384-18cc594cbbd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:42:21.085034 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5e2d1b-2247-4787-8384-18cc594cbbd0 None None] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:42:21.089839 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1807cb-377f-40f6-8d47-601fe149f02a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:42:21.090728 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1807cb-377f-40f6-8d47-601fe149f02a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:42:21.091013 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1807cb-377f-40f6-8d47-601fe149f02a admin admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:42:21.091291 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1807cb-377f-40f6-8d47-601fe149f02a admin admin] PATH_INFO: `/v3/projects/5aaa3579edcb415b97e290757ace1a11` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:42:21.092492 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd1807cb-377f-40f6-8d47-601fe149f02a admin admin] RBAC: Authorizing `identity:delete_project(project_id=5aaa3579edcb415b97e290757ace1a11)` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:42:21.094690 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd1807cb-377f-40f6-8d47-601fe149f02a admin admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:42:21.153506 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-fd1807cb-377f-40f6-8d47-601fe149f02a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5aaa3579edcb415b97e290757ace1a11', 'request_id': 'req-fd1807cb-377f-40f6-8d47-601fe149f02a'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:42:21.161766 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-fd1807cb-377f-40f6-8d47-601fe149f02a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5aaa3579edcb415b97e290757ace1a11', 'request_id': 'req-fd1807cb-377f-40f6-8d47-601fe149f02a'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:42:21.167736 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-fd1807cb-377f-40f6-8d47-601fe149f02a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5aaa3579edcb415b97e290757ace1a11', 'request_id': 'req-fd1807cb-377f-40f6-8d47-601fe149f02a'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:42:21.179732 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-fd1807cb-377f-40f6-8d47-601fe149f02a admin admin] The token cache is being invalidate because project 5aaa3579edcb415b97e290757ace1a11 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80312) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 01 16:42:21.180049 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-fd1807cb-377f-40f6-8d47-601fe149f02a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:42:21.180674 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-fd1807cb-377f-40f6-8d47-601fe149f02a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:42:21.182232 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1136/2253] 10.222.0.57 () {58 vars in 1232 bytes} [Fri Jul 1 16:42:21 2022] DELETE /identity/v3/projects/5aaa3579edcb415b97e290757ace1a11 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 16:42:21.186261 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ae0c4a-c733-4f90-b34d-3437464db19b service neutron] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:42:21.197374 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ae0c4a-c733-4f90-b34d-3437464db19b service neutron] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:42:21.201119 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:42:21.263714 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e490f51d-0364-4e46-bedc-687a73ab88b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:42:21.264506 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e490f51d-0364-4e46-bedc-687a73ab88b4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:42:21.264786 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e490f51d-0364-4e46-bedc-687a73ab88b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:42:21.265722 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e490f51d-0364-4e46-bedc-687a73ab88b4 admin admin] PATH_INFO: `/v3/users/79c7caef9d964b9088aa62d12f83224e` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:42:21.267244 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e490f51d-0364-4e46-bedc-687a73ab88b4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=79c7caef9d964b9088aa62d12f83224e)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:42:21.269313 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e490f51d-0364-4e46-bedc-687a73ab88b4 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:42:21.319847 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-e490f51d-0364-4e46-bedc-687a73ab88b4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '79c7caef9d964b9088aa62d12f83224e'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:42:21.320634 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-e490f51d-0364-4e46-bedc-687a73ab88b4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '79c7caef9d964b9088aa62d12f83224e'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:42:21.321264 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-e490f51d-0364-4e46-bedc-687a73ab88b4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '79c7caef9d964b9088aa62d12f83224e'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:42:21.331004 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-e490f51d-0364-4e46-bedc-687a73ab88b4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '79c7caef9d964b9088aa62d12f83224e'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:42:21.343916 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-e490f51d-0364-4e46-bedc-687a73ab88b4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '79c7caef9d964b9088aa62d12f83224e'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:42:21.356330 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1118/2254] 10.222.0.57 () {58 vars in 1223 bytes} [Fri Jul 1 16:42:21 2022] DELETE /identity/v3/users/79c7caef9d964b9088aa62d12f83224e => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 16:42:21.657221 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1807cb-377f-40f6-8d47-601fe149f02a admin admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:42:21.658822 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-fd1807cb-377f-40f6-8d47-601fe149f02a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:42:21.675247 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1807cb-377f-40f6-8d47-601fe149f02a admin admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:42:21.758110 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38c351f-d768-43ee-b42b-7fbefd80ed49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:42:21.760739 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38c351f-d768-43ee-b42b-7fbefd80ed49 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:42:21.760739 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38c351f-d768-43ee-b42b-7fbefd80ed49 admin admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:42:21.760739 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38c351f-d768-43ee-b42b-7fbefd80ed49 admin admin] PATH_INFO: `/v3/projects/2553a9f2d1b24630be799acce7a93f84` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:42:21.762002 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38c351f-d768-43ee-b42b-7fbefd80ed49 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2553a9f2d1b24630be799acce7a93f84)` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:42:21.765548 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38c351f-d768-43ee-b42b-7fbefd80ed49 admin admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:42:21.828890 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-d38c351f-d768-43ee-b42b-7fbefd80ed49 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2553a9f2d1b24630be799acce7a93f84', 'request_id': 'req-d38c351f-d768-43ee-b42b-7fbefd80ed49'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:42:21.853997 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-d38c351f-d768-43ee-b42b-7fbefd80ed49 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2553a9f2d1b24630be799acce7a93f84', 'request_id': 'req-d38c351f-d768-43ee-b42b-7fbefd80ed49'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:42:21.863257 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:42:21.868082 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-d38c351f-d768-43ee-b42b-7fbefd80ed49 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2553a9f2d1b24630be799acce7a93f84', 'request_id': 'req-d38c351f-d768-43ee-b42b-7fbefd80ed49'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:42:21.904070 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-d38c351f-d768-43ee-b42b-7fbefd80ed49 admin admin] The token cache is being invalidate because project 2553a9f2d1b24630be799acce7a93f84 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80312) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 01 16:42:21.904377 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-d38c351f-d768-43ee-b42b-7fbefd80ed49 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:42:21.905735 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-d38c351f-d768-43ee-b42b-7fbefd80ed49 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:42:21.909440 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1137/2255] 10.222.0.57 () {58 vars in 1232 bytes} [Fri Jul 1 16:42:21 2022] DELETE /identity/v3/projects/2553a9f2d1b24630be799acce7a93f84 => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 16:42:21.913622 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e490f51d-0364-4e46-bedc-687a73ab88b4 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:42:21.916083 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-e490f51d-0364-4e46-bedc-687a73ab88b4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:42:21.935675 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e490f51d-0364-4e46-bedc-687a73ab88b4 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:42:22.029508 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0110eb-4bdd-4aa1-87ab-fd47c99dae5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:42:22.031205 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0110eb-4bdd-4aa1-87ab-fd47c99dae5c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:42:22.032813 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0110eb-4bdd-4aa1-87ab-fd47c99dae5c admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:42:22.032813 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0110eb-4bdd-4aa1-87ab-fd47c99dae5c admin admin] PATH_INFO: `/v3/users/9024df4e5a7241fea4d1a3040c56dc46` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:42:22.040864 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d0110eb-4bdd-4aa1-87ab-fd47c99dae5c admin admin] RBAC: Authorizing `identity:delete_user(user_id=9024df4e5a7241fea4d1a3040c56dc46)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:42:22.043060 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d0110eb-4bdd-4aa1-87ab-fd47c99dae5c admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:42:22.077757 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:42:22.104190 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-4d0110eb-4bdd-4aa1-87ab-fd47c99dae5c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9024df4e5a7241fea4d1a3040c56dc46'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:42:22.104845 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-4d0110eb-4bdd-4aa1-87ab-fd47c99dae5c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9024df4e5a7241fea4d1a3040c56dc46'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:42:22.105709 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-4d0110eb-4bdd-4aa1-87ab-fd47c99dae5c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9024df4e5a7241fea4d1a3040c56dc46'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:42:22.115605 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-4d0110eb-4bdd-4aa1-87ab-fd47c99dae5c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9024df4e5a7241fea4d1a3040c56dc46'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:42:22.131842 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-4d0110eb-4bdd-4aa1-87ab-fd47c99dae5c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9024df4e5a7241fea4d1a3040c56dc46'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:42:22.154194 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1119/2256] 10.222.0.57 () {58 vars in 1223 bytes} [Fri Jul 1 16:42:21 2022] DELETE /identity/v3/users/9024df4e5a7241fea4d1a3040c56dc46 => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 16:42:22.605906 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38c351f-d768-43ee-b42b-7fbefd80ed49 admin admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:42:22.607516 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-d38c351f-d768-43ee-b42b-7fbefd80ed49 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:42:22.623105 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38c351f-d768-43ee-b42b-7fbefd80ed49 admin admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:42:22.684796 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a983e9-1a38-4200-a563-88c7402c00db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:42:22.685662 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a983e9-1a38-4200-a563-88c7402c00db admin admin] REQUEST_METHOD: `DELETE` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:42:22.685943 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a983e9-1a38-4200-a563-88c7402c00db admin admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:42:22.686215 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a983e9-1a38-4200-a563-88c7402c00db admin admin] PATH_INFO: `/v3/projects/bed5cb2d123347d2ae9173d6ae91fbc2` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:42:22.688290 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a983e9-1a38-4200-a563-88c7402c00db admin admin] RBAC: Authorizing `identity:delete_project(project_id=bed5cb2d123347d2ae9173d6ae91fbc2)` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:42:22.691130 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a983e9-1a38-4200-a563-88c7402c00db admin admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:42:22.743858 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-a9a983e9-1a38-4200-a563-88c7402c00db admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bed5cb2d123347d2ae9173d6ae91fbc2', 'request_id': 'req-a9a983e9-1a38-4200-a563-88c7402c00db'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:42:22.759198 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-a9a983e9-1a38-4200-a563-88c7402c00db admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bed5cb2d123347d2ae9173d6ae91fbc2', 'request_id': 'req-a9a983e9-1a38-4200-a563-88c7402c00db'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:42:22.766160 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-a9a983e9-1a38-4200-a563-88c7402c00db admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bed5cb2d123347d2ae9173d6ae91fbc2', 'request_id': 'req-a9a983e9-1a38-4200-a563-88c7402c00db'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:42:22.781405 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-a9a983e9-1a38-4200-a563-88c7402c00db admin admin] The token cache is being invalidate because project bed5cb2d123347d2ae9173d6ae91fbc2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80312) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 01 16:42:22.782030 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-a9a983e9-1a38-4200-a563-88c7402c00db admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:42:22.782796 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-a9a983e9-1a38-4200-a563-88c7402c00db admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:42:22.784824 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1138/2257] 10.222.0.57 () {58 vars in 1232 bytes} [Fri Jul 1 16:42:22 2022] DELETE /identity/v3/projects/bed5cb2d123347d2ae9173d6ae91fbc2 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 16:42:22.789675 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7102d099-b8bf-4360-a6a7-70a91abc1938 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:42:22.790178 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7102d099-b8bf-4360-a6a7-70a91abc1938 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:42:22.790499 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7102d099-b8bf-4360-a6a7-70a91abc1938 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:42:22.810981 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-7102d099-b8bf-4360-a6a7-70a91abc1938 None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:42:22.812318 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-7102d099-b8bf-4360-a6a7-70a91abc1938 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:42:22.895560 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1120/2258] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 1 16:42:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 01 16:42:22.903622 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25963a4-cab6-4bf5-aaff-91db103c25fb None None] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:42:22.904221 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25963a4-cab6-4bf5-aaff-91db103c25fb None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:42:22.904840 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25963a4-cab6-4bf5-aaff-91db103c25fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:42:22.928182 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.auth.core [None req-f25963a4-cab6-4bf5-aaff-91db103c25fb None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80312) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:42:22.929888 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-f25963a4-cab6-4bf5-aaff-91db103c25fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:42:22.958027 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1139/2259] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 1 16:42:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 16:42:22.962340 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7102d099-b8bf-4360-a6a7-70a91abc1938 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:42:22.987165 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7102d099-b8bf-4360-a6a7-70a91abc1938 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:42:22.992405 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67629d58-23ac-4ce1-9b58-a0b98fa1d211 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:42:22.993084 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67629d58-23ac-4ce1-9b58-a0b98fa1d211 admin admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:42:22.993364 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67629d58-23ac-4ce1-9b58-a0b98fa1d211 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:42:22.993635 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67629d58-23ac-4ce1-9b58-a0b98fa1d211 admin admin] PATH_INFO: `/v3/domains` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:42:22.994574 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67629d58-23ac-4ce1-9b58-a0b98fa1d211 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:42:22.995032 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67629d58-23ac-4ce1-9b58-a0b98fa1d211 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:42:22.996864 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67629d58-23ac-4ce1-9b58-a0b98fa1d211 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:42:23.028921 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1121/2260] 10.222.0.57 () {58 vars in 1152 bytes} [Fri Jul 1 16:42:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:42:23.033684 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25963a4-cab6-4bf5-aaff-91db103c25fb None None] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:42:23.043892 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25963a4-cab6-4bf5-aaff-91db103c25fb None None] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:42:23.051363 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f695a1-be40-4e87-a535-bd64665aa7f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:42:23.052358 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f695a1-be40-4e87-a535-bd64665aa7f9 admin admin] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:42:23.052787 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f695a1-be40-4e87-a535-bd64665aa7f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:42:23.053182 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f695a1-be40-4e87-a535-bd64665aa7f9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:42:23.053996 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f695a1-be40-4e87-a535-bd64665aa7f9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:42:23.056168 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f695a1-be40-4e87-a535-bd64665aa7f9 admin admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:42:23.096972 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1140/2261] 10.222.0.57 () {60 vars in 1152 bytes} [Fri Jul 1 16:42:23 2022] POST /identity/v3/projects => generated 371 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 16:42:23.102216 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67629d58-23ac-4ce1-9b58-a0b98fa1d211 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:42:23.115387 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67629d58-23ac-4ce1-9b58-a0b98fa1d211 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:42:23.216923 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9049faba-617e-456b-8d37-588a330ff188 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:42:23.218143 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9049faba-617e-456b-8d37-588a330ff188 admin admin] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:42:23.219174 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9049faba-617e-456b-8d37-588a330ff188 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:42:23.219513 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9049faba-617e-456b-8d37-588a330ff188 admin admin] PATH_INFO: `/v3/users` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:42:23.220287 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9049faba-617e-456b-8d37-588a330ff188 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:42:23.225104 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9049faba-617e-456b-8d37-588a330ff188 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:42:23.271705 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1122/2262] 10.222.0.57 () {60 vars in 1143 bytes} [Fri Jul 1 16:42:23 2022] POST /identity/v3/users => generated 286 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 16:42:23.272931 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f695a1-be40-4e87-a535-bd64665aa7f9 admin admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:42:23.286673 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f695a1-be40-4e87-a535-bd64665aa7f9 admin admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:42:23.292263 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101a27af-2e0b-439f-87b8-3d9e62425b49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:42:23.293530 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101a27af-2e0b-439f-87b8-3d9e62425b49 admin admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:42:23.294118 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101a27af-2e0b-439f-87b8-3d9e62425b49 admin admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:42:23.295289 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101a27af-2e0b-439f-87b8-3d9e62425b49 admin admin] PATH_INFO: `/v3/roles` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:42:23.306296 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-101a27af-2e0b-439f-87b8-3d9e62425b49 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:42:23.309666 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-101a27af-2e0b-439f-87b8-3d9e62425b49 admin admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:42:23.343586 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9049faba-617e-456b-8d37-588a330ff188 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:42:23.344010 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1141/2263] 10.222.0.57 () {58 vars in 1121 bytes} [Fri Jul 1 16:42:23 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 16:42:23.376892 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9049faba-617e-456b-8d37-588a330ff188 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:42:23.387409 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dcd4f44-6256-4294-ac17-af06da185da7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:42:23.388850 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcd4f44-6256-4294-ac17-af06da185da7 admin admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:42:23.389516 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcd4f44-6256-4294-ac17-af06da185da7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:42:23.390734 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcd4f44-6256-4294-ac17-af06da185da7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:42:23.392078 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dcd4f44-6256-4294-ac17-af06da185da7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:42:23.394490 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dcd4f44-6256-4294-ac17-af06da185da7 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:42:23.411554 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1123/2264] 10.222.0.57 () {58 vars in 1121 bytes} [Fri Jul 1 16:42:23 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 01 16:42:23.416338 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101a27af-2e0b-439f-87b8-3d9e62425b49 admin admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:42:23.428661 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101a27af-2e0b-439f-87b8-3d9e62425b49 admin admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:42:23.434638 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a0391e-19f2-41a6-8683-1f675c07dcf0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:42:23.435866 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a0391e-19f2-41a6-8683-1f675c07dcf0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:42:23.436008 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a0391e-19f2-41a6-8683-1f675c07dcf0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:42:23.436299 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a0391e-19f2-41a6-8683-1f675c07dcf0 admin admin] PATH_INFO: `/v3/projects/08b020cde63546ae86d2b2955c01c3c1/users/5d9ef7305def4561b3e14bdcca8f913c/roles/983be40bdab04c478eaa90c6b6a46b14` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:42:23.456286 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a0391e-19f2-41a6-8683-1f675c07dcf0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=08b020cde63546ae86d2b2955c01c3c1, user_id=5d9ef7305def4561b3e14bdcca8f913c, role_id=983be40bdab04c478eaa90c6b6a46b14)` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:42:23.460625 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a0391e-19f2-41a6-8683-1f675c07dcf0 admin admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:42:23.478009 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1142/2265] 10.222.0.57 () {60 vars in 1482 bytes} [Fri Jul 1 16:42:23 2022] PUT /identity/v3/projects/08b020cde63546ae86d2b2955c01c3c1/users/5d9ef7305def4561b3e14bdcca8f913c/roles/983be40bdab04c478eaa90c6b6a46b14 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 01 16:42:23.490076 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e5b0c1-ab94-46ee-a070-ea4761503cd5 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:42:23.491058 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e5b0c1-ab94-46ee-a070-ea4761503cd5 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:42:23.491442 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e5b0c1-ab94-46ee-a070-ea4761503cd5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:42:23.513464 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:42:23.524239 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-d6e5b0c1-ab94-46ee-a070-ea4761503cd5 None None] MFA Rules not processed for user `5d9ef7305def4561b3e14bdcca8f913c`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:42:23.525940 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-d6e5b0c1-ab94-46ee-a070-ea4761503cd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:42:23.630017 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1124/2266] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 1 16:42:23 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 16:42:23.644969 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a0391e-19f2-41a6-8683-1f675c07dcf0 admin admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:42:23.646794 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-16a0391e-19f2-41a6-8683-1f675c07dcf0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:42:23.683469 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:42:23.763143 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a0391e-19f2-41a6-8683-1f675c07dcf0 admin admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:42:23.774274 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87d1ddb-98aa-462c-8866-f7be4fd198fe service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd028715bfea43099e7c6cde429ecf92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0788e6725a84e57a4174284c1845755', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:42:23.775040 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87d1ddb-98aa-462c-8866-f7be4fd198fe service glance] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:42:23.775474 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87d1ddb-98aa-462c-8866-f7be4fd198fe service glance] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:42:23.775754 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87d1ddb-98aa-462c-8866-f7be4fd198fe service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:42:23.786149 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d87d1ddb-98aa-462c-8866-f7be4fd198fe service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:42:23.787692 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d87d1ddb-98aa-462c-8866-f7be4fd198fe service glance] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:42:23.797104 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1143/2267] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 1 16:42:23 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 16:42:32.192477 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e5b0c1-ab94-46ee-a070-ea4761503cd5 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:42:32.193997 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-d6e5b0c1-ab94-46ee-a070-ea4761503cd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:42:32.267566 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e5b0c1-ab94-46ee-a070-ea4761503cd5 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:42:32.272109 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42021f2a-7eae-44c1-8877-1cbe5ffcae3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:42:32.272972 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42021f2a-7eae-44c1-8877-1cbe5ffcae3d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:42:32.273250 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42021f2a-7eae-44c1-8877-1cbe5ffcae3d admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:42:32.273522 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42021f2a-7eae-44c1-8877-1cbe5ffcae3d admin admin] PATH_INFO: `/v3/users/414722f07c4c484a89f9ac5e80c7bab8` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:42:32.274878 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42021f2a-7eae-44c1-8877-1cbe5ffcae3d admin admin] RBAC: Authorizing `identity:delete_user(user_id=414722f07c4c484a89f9ac5e80c7bab8)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:42:32.276792 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42021f2a-7eae-44c1-8877-1cbe5ffcae3d admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:42:32.332863 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-42021f2a-7eae-44c1-8877-1cbe5ffcae3d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '414722f07c4c484a89f9ac5e80c7bab8'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:42:32.333554 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-42021f2a-7eae-44c1-8877-1cbe5ffcae3d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '414722f07c4c484a89f9ac5e80c7bab8'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:42:32.334180 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-42021f2a-7eae-44c1-8877-1cbe5ffcae3d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '414722f07c4c484a89f9ac5e80c7bab8'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:42:32.341638 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-42021f2a-7eae-44c1-8877-1cbe5ffcae3d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '414722f07c4c484a89f9ac5e80c7bab8'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:42:32.355736 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-42021f2a-7eae-44c1-8877-1cbe5ffcae3d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '414722f07c4c484a89f9ac5e80c7bab8'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:42:32.368322 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1125/2268] 10.222.0.57 () {58 vars in 1223 bytes} [Fri Jul 1 16:42:32 2022] DELETE /identity/v3/users/414722f07c4c484a89f9ac5e80c7bab8 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 16:42:32.372942 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707e7ef1-cce2-40ff-9d36-1cae0979e29e None None] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:42:32.373497 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707e7ef1-cce2-40ff-9d36-1cae0979e29e None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:42:32.373497 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707e7ef1-cce2-40ff-9d36-1cae0979e29e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:42:32.392318 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.auth.core [None req-707e7ef1-cce2-40ff-9d36-1cae0979e29e None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80312) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:42:32.393531 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-707e7ef1-cce2-40ff-9d36-1cae0979e29e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:42:32.414621 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fa3df9-894f-4d59-b81c-0880c5e91e3b None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:42:32.414911 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fa3df9-894f-4d59-b81c-0880c5e91e3b None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:42:32.415190 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fa3df9-894f-4d59-b81c-0880c5e91e3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:42:32.433360 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-b7fa3df9-894f-4d59-b81c-0880c5e91e3b None None] MFA Rules not processed for user `1f2a1540974a444f8341d395d6a2b070`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:42:32.437127 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-b7fa3df9-894f-4d59-b81c-0880c5e91e3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:42:32.451392 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:42:32.472995 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1144/2269] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 1 16:42:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 16:42:32.482556 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707e7ef1-cce2-40ff-9d36-1cae0979e29e None None] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:42:32.483993 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-707e7ef1-cce2-40ff-9d36-1cae0979e29e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:42:32.504584 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1126/2270] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 1 16:42:32 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 01 16:42:32.555310 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707e7ef1-cce2-40ff-9d36-1cae0979e29e None None] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:42:32.560958 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f151096a-5857-4a33-ba60-698fcff72e81 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ed200ce29684c5f80354fa9a8638fe2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0788e6725a84e57a4174284c1845755', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:42:32.562790 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f151096a-5857-4a33-ba60-698fcff72e81 service neutron] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:42:32.563378 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f151096a-5857-4a33-ba60-698fcff72e81 service neutron] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:42:32.563675 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f151096a-5857-4a33-ba60-698fcff72e81 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:42:32.574037 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f151096a-5857-4a33-ba60-698fcff72e81 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:42:32.575640 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f151096a-5857-4a33-ba60-698fcff72e81 service neutron] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:42:32.587812 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1145/2271] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 1 16:42:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 16:42:33.305381 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7fa3df9-894f-4d59-b81c-0880c5e91e3b None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:42:33.310491 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-b7fa3df9-894f-4d59-b81c-0880c5e91e3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:42:33.357714 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7fa3df9-894f-4d59-b81c-0880c5e91e3b None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:42:33.366561 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d26ad85-ea96-4d76-9093-1243c702bae9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:42:33.367494 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d26ad85-ea96-4d76-9093-1243c702bae9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:42:33.367783 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d26ad85-ea96-4d76-9093-1243c702bae9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:42:33.368061 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d26ad85-ea96-4d76-9093-1243c702bae9 admin admin] PATH_INFO: `/v3/projects/c30c284bffb441f0b177ffd254af9c5a` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:42:33.370551 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d26ad85-ea96-4d76-9093-1243c702bae9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c30c284bffb441f0b177ffd254af9c5a)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:42:33.376916 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d26ad85-ea96-4d76-9093-1243c702bae9 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:42:33.483089 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-7d26ad85-ea96-4d76-9093-1243c702bae9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c30c284bffb441f0b177ffd254af9c5a', 'request_id': 'req-7d26ad85-ea96-4d76-9093-1243c702bae9'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:42:33.490908 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-7d26ad85-ea96-4d76-9093-1243c702bae9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c30c284bffb441f0b177ffd254af9c5a', 'request_id': 'req-7d26ad85-ea96-4d76-9093-1243c702bae9'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:42:33.522250 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-7d26ad85-ea96-4d76-9093-1243c702bae9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c30c284bffb441f0b177ffd254af9c5a', 'request_id': 'req-7d26ad85-ea96-4d76-9093-1243c702bae9'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:42:33.556017 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-7d26ad85-ea96-4d76-9093-1243c702bae9 admin admin] The token cache is being invalidate because project c30c284bffb441f0b177ffd254af9c5a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80311) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 01 16:42:33.557181 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-7d26ad85-ea96-4d76-9093-1243c702bae9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:42:33.557873 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-7d26ad85-ea96-4d76-9093-1243c702bae9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:42:33.559533 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1127/2272] 10.222.0.57 () {58 vars in 1232 bytes} [Fri Jul 1 16:42:33 2022] DELETE /identity/v3/projects/c30c284bffb441f0b177ffd254af9c5a => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 16:42:33.567897 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa088dc1-4e73-401c-b2d0-ce7d23382bbb None None] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:42:33.568919 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa088dc1-4e73-401c-b2d0-ce7d23382bbb None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:42:33.568919 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa088dc1-4e73-401c-b2d0-ce7d23382bbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:42:33.589229 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.auth.core [None req-fa088dc1-4e73-401c-b2d0-ce7d23382bbb None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80312) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:42:33.590692 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-fa088dc1-4e73-401c-b2d0-ce7d23382bbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:42:33.727717 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1146/2273] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 1 16:42:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 16:42:33.736754 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb1c836-dd6d-46b7-8c18-778201b03dd0 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:42:33.737937 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb1c836-dd6d-46b7-8c18-778201b03dd0 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:42:33.738296 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb1c836-dd6d-46b7-8c18-778201b03dd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:42:33.764788 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-9eb1c836-dd6d-46b7-8c18-778201b03dd0 None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:42:33.767920 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-9eb1c836-dd6d-46b7-8c18-778201b03dd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:42:33.779718 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1128/2274] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 1 16:42:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 16:42:33.780936 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa088dc1-4e73-401c-b2d0-ce7d23382bbb None None] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:42:33.805477 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=100 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:42:33.812399 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa088dc1-4e73-401c-b2d0-ce7d23382bbb None None] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:42:33.823281 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71b495a-425c-4305-901d-876e1731e547 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:42:33.823967 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71b495a-425c-4305-901d-876e1731e547 admin admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:42:33.824253 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71b495a-425c-4305-901d-876e1731e547 admin admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:42:33.824582 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71b495a-425c-4305-901d-876e1731e547 admin admin] PATH_INFO: `/v3/domains` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:42:33.826535 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d71b495a-425c-4305-901d-876e1731e547 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80312) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:42:33.827004 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d71b495a-425c-4305-901d-876e1731e547 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:42:33.830334 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d71b495a-425c-4305-901d-876e1731e547 admin admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:42:33.864856 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1147/2275] 10.222.0.57 () {58 vars in 1152 bytes} [Fri Jul 1 16:42:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:42:33.873456 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb1c836-dd6d-46b7-8c18-778201b03dd0 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:42:33.885773 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb1c836-dd6d-46b7-8c18-778201b03dd0 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:42:33.890882 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04450a49-cbf9-4ca7-86c8-db9b4f671a53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:42:33.891659 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04450a49-cbf9-4ca7-86c8-db9b4f671a53 admin admin] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:42:33.892022 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04450a49-cbf9-4ca7-86c8-db9b4f671a53 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:42:33.892569 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04450a49-cbf9-4ca7-86c8-db9b4f671a53 admin admin] PATH_INFO: `/v3/projects` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:42:33.893340 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04450a49-cbf9-4ca7-86c8-db9b4f671a53 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:42:33.895408 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04450a49-cbf9-4ca7-86c8-db9b4f671a53 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:42:33.937386 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1129/2276] 10.222.0.57 () {60 vars in 1152 bytes} [Fri Jul 1 16:42:33 2022] POST /identity/v3/projects => generated 371 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 16:42:33.942915 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71b495a-425c-4305-901d-876e1731e547 admin admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:42:33.954575 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71b495a-425c-4305-901d-876e1731e547 admin admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:42:34.033787 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a027cf-e697-4031-8db3-b01df465740a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:42:34.034583 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a027cf-e697-4031-8db3-b01df465740a admin admin] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:42:34.034941 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a027cf-e697-4031-8db3-b01df465740a admin admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:42:34.035453 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a027cf-e697-4031-8db3-b01df465740a admin admin] PATH_INFO: `/v3/users` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:42:34.036320 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a027cf-e697-4031-8db3-b01df465740a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:42:34.038427 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a027cf-e697-4031-8db3-b01df465740a admin admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:42:34.060799 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1148/2277] 10.222.0.57 () {60 vars in 1143 bytes} [Fri Jul 1 16:42:33 2022] POST /identity/v3/users => generated 286 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 01 16:42:34.065540 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04450a49-cbf9-4ca7-86c8-db9b4f671a53 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:42:34.076235 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04450a49-cbf9-4ca7-86c8-db9b4f671a53 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:42:34.082076 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b59c232-d20c-4d53-92ff-15b7556280c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:42:34.082796 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b59c232-d20c-4d53-92ff-15b7556280c7 admin admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:42:34.083081 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b59c232-d20c-4d53-92ff-15b7556280c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:42:34.083361 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b59c232-d20c-4d53-92ff-15b7556280c7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:42:34.084100 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b59c232-d20c-4d53-92ff-15b7556280c7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:42:34.085969 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b59c232-d20c-4d53-92ff-15b7556280c7 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:42:34.101607 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1130/2278] 10.222.0.57 () {58 vars in 1121 bytes} [Fri Jul 1 16:42:34 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 16:42:34.107330 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a027cf-e697-4031-8db3-b01df465740a admin admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:42:34.117948 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a027cf-e697-4031-8db3-b01df465740a admin admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:42:34.123226 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c407a4d-624e-47a2-a26b-520a0eb91efe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:42:34.123969 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c407a4d-624e-47a2-a26b-520a0eb91efe admin admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:42:34.124703 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c407a4d-624e-47a2-a26b-520a0eb91efe admin admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:42:34.124703 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c407a4d-624e-47a2-a26b-520a0eb91efe admin admin] PATH_INFO: `/v3/roles` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:42:34.125244 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c407a4d-624e-47a2-a26b-520a0eb91efe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:42:34.126898 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c407a4d-624e-47a2-a26b-520a0eb91efe admin admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:42:34.138520 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1149/2279] 10.222.0.57 () {58 vars in 1121 bytes} [Fri Jul 1 16:42:34 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 16:42:34.142918 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b59c232-d20c-4d53-92ff-15b7556280c7 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:42:34.152981 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b59c232-d20c-4d53-92ff-15b7556280c7 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:42:34.158833 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a8950d-7c05-45fa-a2db-3eda5eefaeb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:42:34.159859 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a8950d-7c05-45fa-a2db-3eda5eefaeb1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:42:34.160151 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a8950d-7c05-45fa-a2db-3eda5eefaeb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:42:34.160431 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a8950d-7c05-45fa-a2db-3eda5eefaeb1 admin admin] PATH_INFO: `/v3/projects/1eb9237b6e8a4b5ca032f0664d09f28e/users/5b62962ac2a14ae997fb637e80129beb/roles/983be40bdab04c478eaa90c6b6a46b14` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:42:34.177386 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a8950d-7c05-45fa-a2db-3eda5eefaeb1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1eb9237b6e8a4b5ca032f0664d09f28e, user_id=5b62962ac2a14ae997fb637e80129beb, role_id=983be40bdab04c478eaa90c6b6a46b14)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:42:34.180353 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a8950d-7c05-45fa-a2db-3eda5eefaeb1 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:42:34.198540 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1131/2280] 10.222.0.57 () {60 vars in 1482 bytes} [Fri Jul 1 16:42:34 2022] PUT /identity/v3/projects/1eb9237b6e8a4b5ca032f0664d09f28e/users/5b62962ac2a14ae997fb637e80129beb/roles/983be40bdab04c478eaa90c6b6a46b14 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 16:42:34.204257 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ede42a-8a49-4886-9590-2b25a51258cf None None] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:42:34.204487 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ede42a-8a49-4886-9590-2b25a51258cf None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:42:34.204782 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ede42a-8a49-4886-9590-2b25a51258cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:42:34.234220 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.auth.core [None req-e3ede42a-8a49-4886-9590-2b25a51258cf None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80312) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:42:34.235756 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-e3ede42a-8a49-4886-9590-2b25a51258cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:42:34.360769 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1150/2281] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 1 16:42:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 16:42:34.377598 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d537627-8894-45ba-be12-1547d62982e3 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:42:34.378196 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d537627-8894-45ba-be12-1547d62982e3 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:42:34.378196 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d537627-8894-45ba-be12-1547d62982e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:42:34.402432 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-6d537627-8894-45ba-be12-1547d62982e3 None None] MFA Rules not processed for user `c4de1b1eeaed4fc5ae7200b2c7a90957`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:42:34.404001 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-6d537627-8894-45ba-be12-1547d62982e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:42:34.414382 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ede42a-8a49-4886-9590-2b25a51258cf None None] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:42:34.415875 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-e3ede42a-8a49-4886-9590-2b25a51258cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:42:34.499295 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:42:34.553113 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1132/2282] 10.222.0.57 () {58 vars in 1009 bytes} [Fri Jul 1 16:42:34 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 16:42:34.569673 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ede42a-8a49-4886-9590-2b25a51258cf None None] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:42:34.578261 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d537627-8894-45ba-be12-1547d62982e3 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:42:34.579800 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd7fc68-af78-41c8-865c-aed348999456 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ed200ce29684c5f80354fa9a8638fe2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0788e6725a84e57a4174284c1845755', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:42:34.579986 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-6d537627-8894-45ba-be12-1547d62982e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:42:34.580974 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd7fc68-af78-41c8-865c-aed348999456 service neutron] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:42:34.581553 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd7fc68-af78-41c8-865c-aed348999456 service neutron] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:42:34.582011 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd7fc68-af78-41c8-865c-aed348999456 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:42:34.604458 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd7fc68-af78-41c8-865c-aed348999456 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:42:34.613153 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd7fc68-af78-41c8-865c-aed348999456 service neutron] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:42:34.629915 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1151/2283] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 1 16:42:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 216 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 16:42:34.703214 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d537627-8894-45ba-be12-1547d62982e3 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:42:34.708996 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3229f006-b8e4-4107-9ea7-67e94a5defae service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '859fe410c2d2450fb803c6244839f27c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0788e6725a84e57a4174284c1845755', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:42:34.710491 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3229f006-b8e4-4107-9ea7-67e94a5defae service swift] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:42:34.710491 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3229f006-b8e4-4107-9ea7-67e94a5defae service swift] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:42:34.710887 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3229f006-b8e4-4107-9ea7-67e94a5defae service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:42:34.722118 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3229f006-b8e4-4107-9ea7-67e94a5defae service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:42:34.723925 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3229f006-b8e4-4107-9ea7-67e94a5defae service swift] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:42:34.733547 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1133/2284] 10.222.0.57 () {60 vars in 1384 bytes} [Fri Jul 1 16:42:34 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 01 16:42:44.140608 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ff2808-1b70-4d57-bd01-1eebc8eaa4f2 None None] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:42:44.141232 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ff2808-1b70-4d57-bd01-1eebc8eaa4f2 None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:42:44.141232 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ff2808-1b70-4d57-bd01-1eebc8eaa4f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:42:44.180414 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.auth.core [None req-63ff2808-1b70-4d57-bd01-1eebc8eaa4f2 None None] MFA Rules not processed for user `71c383c04f9543d8ab97a6c0db268ca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80312) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:42:44.181681 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-63ff2808-1b70-4d57-bd01-1eebc8eaa4f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:42:44.197506 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:42:44.237488 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3229f006-b8e4-4107-9ea7-67e94a5defae service swift] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:42:44.257238 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3229f006-b8e4-4107-9ea7-67e94a5defae service swift] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:42:44.285771 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c44adb0-d068-4271-96f1-ac643915e36c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ed200ce29684c5f80354fa9a8638fe2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0788e6725a84e57a4174284c1845755', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:42:44.286503 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c44adb0-d068-4271-96f1-ac643915e36c service neutron] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:42:44.286798 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c44adb0-d068-4271-96f1-ac643915e36c service neutron] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:42:44.287078 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c44adb0-d068-4271-96f1-ac643915e36c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:42:44.288990 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-2c44adb0-d068-4271-96f1-ac643915e36c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:42:44.365639 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1152/2285] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 1 16:42:44 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 16:42:44.536348 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c44adb0-d068-4271-96f1-ac643915e36c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:42:44.538244 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c44adb0-d068-4271-96f1-ac643915e36c service neutron] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:42:44.554177 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1134/2286] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 1 16:42:44 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 16:42:45.420188 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e551ea-4d7c-4fae-a47c-4709e933e141 None None] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:42:45.422038 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e551ea-4d7c-4fae-a47c-4709e933e141 None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:42:45.422739 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e551ea-4d7c-4fae-a47c-4709e933e141 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:42:45.500873 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.auth.core [None req-02e551ea-4d7c-4fae-a47c-4709e933e141 None None] MFA Rules not processed for user `5b62962ac2a14ae997fb637e80129beb`. Rule list: `[]` (Enabled: `True`). {{(pid=80312) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:42:45.507940 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-02e551ea-4d7c-4fae-a47c-4709e933e141 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:42:45.643012 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1153/2287] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 1 16:42:45 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 16:42:45.647313 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c44adb0-d068-4271-96f1-ac643915e36c service neutron] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:42:45.763776 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c44adb0-d068-4271-96f1-ac643915e36c service neutron] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:42:45.782058 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cbe8350-81d1-4206-b28f-19c63f4d691d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:42:45.785252 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbe8350-81d1-4206-b28f-19c63f4d691d admin admin] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:42:45.788571 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbe8350-81d1-4206-b28f-19c63f4d691d admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:42:45.791093 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbe8350-81d1-4206-b28f-19c63f4d691d admin admin] PATH_INFO: `/v3/projects` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:42:45.792448 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cbe8350-81d1-4206-b28f-19c63f4d691d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:42:45.799313 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cbe8350-81d1-4206-b28f-19c63f4d691d admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:42:45.899773 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:42:45.912734 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1135/2288] 10.222.0.57 () {60 vars in 1152 bytes} [Fri Jul 1 16:42:45 2022] POST /identity/v3/projects => generated 369 bytes in 266 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 16:42:45.916528 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e551ea-4d7c-4fae-a47c-4709e933e141 None None] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:42:45.965470 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e551ea-4d7c-4fae-a47c-4709e933e141 None None] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:42:46.085102 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a811fe-a035-4080-a14e-7bd3202e0d1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:42:46.085894 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a811fe-a035-4080-a14e-7bd3202e0d1a admin admin] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:42:46.086373 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a811fe-a035-4080-a14e-7bd3202e0d1a admin admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:42:46.086822 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a811fe-a035-4080-a14e-7bd3202e0d1a admin admin] PATH_INFO: `/v3/users` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:42:46.087687 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a811fe-a035-4080-a14e-7bd3202e0d1a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:42:46.089791 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a811fe-a035-4080-a14e-7bd3202e0d1a admin admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:42:46.143215 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1154/2289] 10.222.0.57 () {60 vars in 1143 bytes} [Fri Jul 1 16:42:45 2022] POST /identity/v3/users => generated 291 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 01 16:42:46.143530 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cbe8350-81d1-4206-b28f-19c63f4d691d admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:42:46.181417 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cbe8350-81d1-4206-b28f-19c63f4d691d admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:42:46.194371 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e1818b-16ba-41a9-a1e7-d2e59257d7ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:42:46.195108 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e1818b-16ba-41a9-a1e7-d2e59257d7ff admin admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:42:46.195419 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e1818b-16ba-41a9-a1e7-d2e59257d7ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:42:46.195705 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e1818b-16ba-41a9-a1e7-d2e59257d7ff admin admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:42:46.196485 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e1818b-16ba-41a9-a1e7-d2e59257d7ff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:42:46.198158 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e1818b-16ba-41a9-a1e7-d2e59257d7ff admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:42:46.215019 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1136/2290] 10.222.0.57 () {58 vars in 1121 bytes} [Fri Jul 1 16:42:46 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 16:42:46.216263 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a811fe-a035-4080-a14e-7bd3202e0d1a admin admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:42:46.229678 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a811fe-a035-4080-a14e-7bd3202e0d1a admin admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:42:46.247998 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5544677-ca2a-4f5c-9608-23bd01335956 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:42:46.250365 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5544677-ca2a-4f5c-9608-23bd01335956 admin admin] REQUEST_METHOD: `PUT` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:42:46.251810 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5544677-ca2a-4f5c-9608-23bd01335956 admin admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:42:46.252073 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5544677-ca2a-4f5c-9608-23bd01335956 admin admin] PATH_INFO: `/v3/projects/03752bab4db8497a911823c2ef95017f/users/1cc31ac5f57b46d1ad501cfa05a6cbbc/roles/84ee739146934c1ab32c501ff8093644` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:42:46.298024 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5544677-ca2a-4f5c-9608-23bd01335956 admin admin] RBAC: Authorizing `identity:create_grant(project_id=03752bab4db8497a911823c2ef95017f, user_id=1cc31ac5f57b46d1ad501cfa05a6cbbc, role_id=84ee739146934c1ab32c501ff8093644)` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:42:46.313097 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5544677-ca2a-4f5c-9608-23bd01335956 admin admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:42:46.351213 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1155/2291] 10.222.0.57 () {60 vars in 1482 bytes} [Fri Jul 1 16:42:46 2022] PUT /identity/v3/projects/03752bab4db8497a911823c2ef95017f/users/1cc31ac5f57b46d1ad501cfa05a6cbbc/roles/84ee739146934c1ab32c501ff8093644 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 16:42:46.355159 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e1818b-16ba-41a9-a1e7-d2e59257d7ff admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:42:46.370551 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e1818b-16ba-41a9-a1e7-d2e59257d7ff admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:42:46.550770 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaafca78-bb7c-4467-aace-dd124fee7760 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:42:46.551723 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaafca78-bb7c-4467-aace-dd124fee7760 admin admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:42:46.552169 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaafca78-bb7c-4467-aace-dd124fee7760 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:42:46.552600 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaafca78-bb7c-4467-aace-dd124fee7760 admin admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:42:46.554272 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaafca78-bb7c-4467-aace-dd124fee7760 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:42:46.556302 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaafca78-bb7c-4467-aace-dd124fee7760 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:42:46.591088 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1137/2292] 10.222.0.57 () {58 vars in 1121 bytes} [Fri Jul 1 16:42:46 2022] GET /identity/v3/roles => generated 1420 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 16:42:46.595992 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5544677-ca2a-4f5c-9608-23bd01335956 admin admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:42:46.613953 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:42:46.623002 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5544677-ca2a-4f5c-9608-23bd01335956 admin admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:42:46.634260 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc10ac19-e19d-4f6a-ae38-596143845341 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:42:46.635254 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc10ac19-e19d-4f6a-ae38-596143845341 admin admin] REQUEST_METHOD: `PUT` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:42:46.635550 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc10ac19-e19d-4f6a-ae38-596143845341 admin admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:42:46.636000 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc10ac19-e19d-4f6a-ae38-596143845341 admin admin] PATH_INFO: `/v3/projects/03752bab4db8497a911823c2ef95017f/users/1cc31ac5f57b46d1ad501cfa05a6cbbc/roles/84ee739146934c1ab32c501ff8093644` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:42:46.638912 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc10ac19-e19d-4f6a-ae38-596143845341 admin admin] RBAC: Authorizing `identity:create_grant(project_id=03752bab4db8497a911823c2ef95017f, user_id=1cc31ac5f57b46d1ad501cfa05a6cbbc, role_id=84ee739146934c1ab32c501ff8093644)` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:42:46.642040 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc10ac19-e19d-4f6a-ae38-596143845341 admin admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:42:46.680603 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1156/2293] 10.222.0.57 () {60 vars in 1482 bytes} [Fri Jul 1 16:42:46 2022] PUT /identity/v3/projects/03752bab4db8497a911823c2ef95017f/users/1cc31ac5f57b46d1ad501cfa05a6cbbc/roles/84ee739146934c1ab32c501ff8093644 => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 01 16:42:56.573450 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=34 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:42:56.667143 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=3,keystone:INSERT=1 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:42:59.783529 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18441856-c375-41b0-8c19-c3e0c42d7cab None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:42:59.784038 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18441856-c375-41b0-8c19-c3e0c42d7cab None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:42:59.784119 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18441856-c375-41b0-8c19-c3e0c42d7cab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:42:59.820176 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-18441856-c375-41b0-8c19-c3e0c42d7cab None None] MFA Rules not processed for user `1cc31ac5f57b46d1ad501cfa05a6cbbc`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:42:59.821397 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-18441856-c375-41b0-8c19-c3e0c42d7cab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:42:59.850300 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc10ac19-e19d-4f6a-ae38-596143845341 admin admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:42:59.854133 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-fc10ac19-e19d-4f6a-ae38-596143845341 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:42:59.940421 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc10ac19-e19d-4f6a-ae38-596143845341 admin admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:42:59.948873 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75eea77a-ee87-40f5-a6bc-cd3c72c19a36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:42:59.949770 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75eea77a-ee87-40f5-a6bc-cd3c72c19a36 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:42:59.950056 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75eea77a-ee87-40f5-a6bc-cd3c72c19a36 admin admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:42:59.950329 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75eea77a-ee87-40f5-a6bc-cd3c72c19a36 admin admin] PATH_INFO: `/v3/users/5d9ef7305def4561b3e14bdcca8f913c` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:42:59.960399 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1138/2294] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 1 16:42:59 2022] POST /identity/v3/auth/tokens => generated 3303 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 16:42:59.960526 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75eea77a-ee87-40f5-a6bc-cd3c72c19a36 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5d9ef7305def4561b3e14bdcca8f913c)` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:42:59.960526 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75eea77a-ee87-40f5-a6bc-cd3c72c19a36 admin admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:43:00.047120 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-75eea77a-ee87-40f5-a6bc-cd3c72c19a36 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5d9ef7305def4561b3e14bdcca8f913c'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:43:00.075535 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-75eea77a-ee87-40f5-a6bc-cd3c72c19a36 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5d9ef7305def4561b3e14bdcca8f913c'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:43:00.092721 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-75eea77a-ee87-40f5-a6bc-cd3c72c19a36 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5d9ef7305def4561b3e14bdcca8f913c'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:43:00.093716 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-75eea77a-ee87-40f5-a6bc-cd3c72c19a36 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5d9ef7305def4561b3e14bdcca8f913c'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:43:00.095514 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-75eea77a-ee87-40f5-a6bc-cd3c72c19a36 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5d9ef7305def4561b3e14bdcca8f913c'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:43:00.114759 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1157/2295] 10.222.0.57 () {58 vars in 1223 bytes} [Fri Jul 1 16:42:59 2022] DELETE /identity/v3/users/5d9ef7305def4561b3e14bdcca8f913c => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 16:43:00.118576 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a449f0cc-6248-4434-b99b-27c5caa3c10c None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:43:00.118869 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a449f0cc-6248-4434-b99b-27c5caa3c10c None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:43:00.119142 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a449f0cc-6248-4434-b99b-27c5caa3c10c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:43:00.153949 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-a449f0cc-6248-4434-b99b-27c5caa3c10c None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:43:00.155356 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-a449f0cc-6248-4434-b99b-27c5caa3c10c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:43:00.243896 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1139/2296] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 1 16:43:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 16:43:00.259134 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75eea77a-ee87-40f5-a6bc-cd3c72c19a36 admin admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:43:00.263166 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-75eea77a-ee87-40f5-a6bc-cd3c72c19a36 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:43:00.318411 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a449f0cc-6248-4434-b99b-27c5caa3c10c None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:43:00.318411 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-a449f0cc-6248-4434-b99b-27c5caa3c10c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:43:00.424730 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:43:00.443022 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75eea77a-ee87-40f5-a6bc-cd3c72c19a36 admin admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:43:00.451281 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c897156e-887d-49c2-ac3e-aeb1e9eb96ee service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ed200ce29684c5f80354fa9a8638fe2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0788e6725a84e57a4174284c1845755', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:43:00.452880 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c897156e-887d-49c2-ac3e-aeb1e9eb96ee service neutron] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:43:00.453321 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c897156e-887d-49c2-ac3e-aeb1e9eb96ee service neutron] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:43:00.453768 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c897156e-887d-49c2-ac3e-aeb1e9eb96ee service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:43:00.464293 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a449f0cc-6248-4434-b99b-27c5caa3c10c None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:43:00.471711 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098442b3-7d6d-475f-b44c-695f155a24ed service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a50f745b31941b7a7265c0fc852f48e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0788e6725a84e57a4174284c1845755', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:43:00.472496 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098442b3-7d6d-475f-b44c-695f155a24ed service nova] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:43:00.472858 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098442b3-7d6d-475f-b44c-695f155a24ed service nova] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:43:00.473215 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098442b3-7d6d-475f-b44c-695f155a24ed service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:43:00.475170 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-098442b3-7d6d-475f-b44c-695f155a24ed service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:43:00.479135 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c897156e-887d-49c2-ac3e-aeb1e9eb96ee service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:43:00.480780 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c897156e-887d-49c2-ac3e-aeb1e9eb96ee service neutron] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:43:00.497345 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:43:00.500247 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1158/2297] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 1 16:43:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 242 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 01 16:43:00.570023 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-098442b3-7d6d-475f-b44c-695f155a24ed service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:43:00.573467 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-098442b3-7d6d-475f-b44c-695f155a24ed service nova] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:43:00.610218 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1140/2298] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 1 16:43:00 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 289 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 01 16:43:01.019369 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c897156e-887d-49c2-ac3e-aeb1e9eb96ee service neutron] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:43:01.020808 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-c897156e-887d-49c2-ac3e-aeb1e9eb96ee service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:43:01.035993 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c897156e-887d-49c2-ac3e-aeb1e9eb96ee service neutron] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:43:01.041661 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61d02a2-1756-49d0-98e6-b81327051f51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:43:01.043755 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61d02a2-1756-49d0-98e6-b81327051f51 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:43:01.043755 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61d02a2-1756-49d0-98e6-b81327051f51 admin admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:43:01.043755 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61d02a2-1756-49d0-98e6-b81327051f51 admin admin] PATH_INFO: `/v3/projects/08b020cde63546ae86d2b2955c01c3c1` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:43:01.046132 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f61d02a2-1756-49d0-98e6-b81327051f51 admin admin] RBAC: Authorizing `identity:delete_project(project_id=08b020cde63546ae86d2b2955c01c3c1)` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:43:01.048433 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f61d02a2-1756-49d0-98e6-b81327051f51 admin admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:43:01.112318 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-f61d02a2-1756-49d0-98e6-b81327051f51 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '08b020cde63546ae86d2b2955c01c3c1', 'request_id': 'req-f61d02a2-1756-49d0-98e6-b81327051f51'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:43:01.121066 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-f61d02a2-1756-49d0-98e6-b81327051f51 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '08b020cde63546ae86d2b2955c01c3c1', 'request_id': 'req-f61d02a2-1756-49d0-98e6-b81327051f51'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:43:01.130998 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-f61d02a2-1756-49d0-98e6-b81327051f51 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '08b020cde63546ae86d2b2955c01c3c1', 'request_id': 'req-f61d02a2-1756-49d0-98e6-b81327051f51'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:43:01.146807 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-f61d02a2-1756-49d0-98e6-b81327051f51 admin admin] The token cache is being invalidate because project 08b020cde63546ae86d2b2955c01c3c1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80312) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 01 16:43:01.147410 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-f61d02a2-1756-49d0-98e6-b81327051f51 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:43:01.147785 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-f61d02a2-1756-49d0-98e6-b81327051f51 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:43:01.149416 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1159/2299] 10.222.0.57 () {58 vars in 1232 bytes} [Fri Jul 1 16:43:01 2022] DELETE /identity/v3/projects/08b020cde63546ae86d2b2955c01c3c1 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 16:43:01.155698 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee401879-d8c0-43d6-8fc3-b5aafb46a6c3 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:43:01.156264 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee401879-d8c0-43d6-8fc3-b5aafb46a6c3 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:43:01.156476 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee401879-d8c0-43d6-8fc3-b5aafb46a6c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:43:01.211423 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-ee401879-d8c0-43d6-8fc3-b5aafb46a6c3 None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:43:01.212642 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-ee401879-d8c0-43d6-8fc3-b5aafb46a6c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:43:01.293573 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1141/2300] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 1 16:43:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 01 16:43:01.302139 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342e6970-3be1-4337-8c84-4253a6cd3ec4 None None] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:43:01.302461 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342e6970-3be1-4337-8c84-4253a6cd3ec4 None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:43:01.302745 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342e6970-3be1-4337-8c84-4253a6cd3ec4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:43:01.325178 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.auth.core [None req-342e6970-3be1-4337-8c84-4253a6cd3ec4 None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80312) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:43:01.326563 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-342e6970-3be1-4337-8c84-4253a6cd3ec4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:43:01.344409 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee401879-d8c0-43d6-8fc3-b5aafb46a6c3 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:43:01.350614 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1160/2301] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 1 16:43:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 16:43:01.355511 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee401879-d8c0-43d6-8fc3-b5aafb46a6c3 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:43:01.364158 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec679d33-9820-40e1-84de-0d022382a383 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:43:01.365396 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec679d33-9820-40e1-84de-0d022382a383 admin admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:43:01.365765 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec679d33-9820-40e1-84de-0d022382a383 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:43:01.366121 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec679d33-9820-40e1-84de-0d022382a383 admin admin] PATH_INFO: `/v3/domains` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:43:01.367352 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec679d33-9820-40e1-84de-0d022382a383 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:43:01.367998 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec679d33-9820-40e1-84de-0d022382a383 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:43:01.369894 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec679d33-9820-40e1-84de-0d022382a383 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:43:01.392254 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1142/2302] 10.222.0.57 () {58 vars in 1152 bytes} [Fri Jul 1 16:43:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:43:01.396435 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342e6970-3be1-4337-8c84-4253a6cd3ec4 None None] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:43:01.408238 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342e6970-3be1-4337-8c84-4253a6cd3ec4 None None] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:43:01.415196 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313a70c0-f390-46b5-b1ff-fe0963d0f7e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:43:01.416117 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313a70c0-f390-46b5-b1ff-fe0963d0f7e1 admin admin] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:43:01.416586 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313a70c0-f390-46b5-b1ff-fe0963d0f7e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:43:01.417155 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313a70c0-f390-46b5-b1ff-fe0963d0f7e1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:43:01.418335 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-313a70c0-f390-46b5-b1ff-fe0963d0f7e1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:43:01.420856 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-313a70c0-f390-46b5-b1ff-fe0963d0f7e1 admin admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:43:01.461468 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1161/2303] 10.222.0.57 () {60 vars in 1152 bytes} [Fri Jul 1 16:43:01 2022] POST /identity/v3/projects => generated 365 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 16:43:01.465347 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec679d33-9820-40e1-84de-0d022382a383 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:43:01.477337 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec679d33-9820-40e1-84de-0d022382a383 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:43:01.559368 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abececb5-2c96-4aab-be4e-6653cf1f093b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:43:01.561630 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abececb5-2c96-4aab-be4e-6653cf1f093b admin admin] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:43:01.562375 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abececb5-2c96-4aab-be4e-6653cf1f093b admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:43:01.562940 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abececb5-2c96-4aab-be4e-6653cf1f093b admin admin] PATH_INFO: `/v3/users` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:43:01.564398 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abececb5-2c96-4aab-be4e-6653cf1f093b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:43:01.571372 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abececb5-2c96-4aab-be4e-6653cf1f093b admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:43:01.627444 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313a70c0-f390-46b5-b1ff-fe0963d0f7e1 admin admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:43:01.629359 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1143/2304] 10.222.0.57 () {60 vars in 1143 bytes} [Fri Jul 1 16:43:01 2022] POST /identity/v3/users => generated 283 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 16:43:01.641145 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313a70c0-f390-46b5-b1ff-fe0963d0f7e1 admin admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:43:01.647019 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b276964d-7902-4821-b84f-251605d4870d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:43:01.648033 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b276964d-7902-4821-b84f-251605d4870d admin admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:43:01.648033 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b276964d-7902-4821-b84f-251605d4870d admin admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:43:01.648791 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b276964d-7902-4821-b84f-251605d4870d admin admin] PATH_INFO: `/v3/roles` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:43:01.649559 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b276964d-7902-4821-b84f-251605d4870d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:43:01.651230 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b276964d-7902-4821-b84f-251605d4870d admin admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:43:01.665412 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1162/2305] 10.222.0.57 () {58 vars in 1121 bytes} [Fri Jul 1 16:43:01 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 01 16:43:01.669324 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abececb5-2c96-4aab-be4e-6653cf1f093b admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:43:01.691554 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abececb5-2c96-4aab-be4e-6653cf1f093b admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:43:01.705213 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e88883-3127-4aa4-9054-29cf8ee84676 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:43:01.707278 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e88883-3127-4aa4-9054-29cf8ee84676 admin admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:43:01.707642 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e88883-3127-4aa4-9054-29cf8ee84676 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:43:01.708104 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e88883-3127-4aa4-9054-29cf8ee84676 admin admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:43:01.709091 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e88883-3127-4aa4-9054-29cf8ee84676 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:43:01.713520 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e88883-3127-4aa4-9054-29cf8ee84676 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:43:01.725977 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:43:01.755363 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1144/2306] 10.222.0.57 () {58 vars in 1121 bytes} [Fri Jul 1 16:43:01 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 16:43:01.759146 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b276964d-7902-4821-b84f-251605d4870d admin admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:43:01.775933 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b276964d-7902-4821-b84f-251605d4870d admin admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:43:01.798608 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560a36c5-cdeb-4872-92be-dc52620ece7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:43:01.800937 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560a36c5-cdeb-4872-92be-dc52620ece7e admin admin] REQUEST_METHOD: `PUT` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:43:01.801391 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560a36c5-cdeb-4872-92be-dc52620ece7e admin admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:43:01.801839 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560a36c5-cdeb-4872-92be-dc52620ece7e admin admin] PATH_INFO: `/v3/projects/426eb4d841c2477295ef705b1eba6d48/users/87ce0bd7ff6f4e7a9a0b1c9b869f31c5/roles/983be40bdab04c478eaa90c6b6a46b14` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:43:01.830400 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-560a36c5-cdeb-4872-92be-dc52620ece7e admin admin] RBAC: Authorizing `identity:create_grant(project_id=426eb4d841c2477295ef705b1eba6d48, user_id=87ce0bd7ff6f4e7a9a0b1c9b869f31c5, role_id=983be40bdab04c478eaa90c6b6a46b14)` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:43:01.833476 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-560a36c5-cdeb-4872-92be-dc52620ece7e admin admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:43:01.869834 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1163/2307] 10.222.0.57 () {60 vars in 1482 bytes} [Fri Jul 1 16:43:01 2022] PUT /identity/v3/projects/426eb4d841c2477295ef705b1eba6d48/users/87ce0bd7ff6f4e7a9a0b1c9b869f31c5/roles/983be40bdab04c478eaa90c6b6a46b14 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 16:43:01.872233 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194f2e29-dad5-42be-8e6d-3329d2d1aa2a None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:43:01.874041 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194f2e29-dad5-42be-8e6d-3329d2d1aa2a None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:43:01.874041 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194f2e29-dad5-42be-8e6d-3329d2d1aa2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:43:01.922994 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-194f2e29-dad5-42be-8e6d-3329d2d1aa2a None None] MFA Rules not processed for user `87ce0bd7ff6f4e7a9a0b1c9b869f31c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:43:01.926575 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-194f2e29-dad5-42be-8e6d-3329d2d1aa2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:43:02.046809 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1145/2308] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 1 16:43:01 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 16:43:02.051766 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560a36c5-cdeb-4872-92be-dc52620ece7e admin admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:43:02.055122 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-560a36c5-cdeb-4872-92be-dc52620ece7e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:43:02.152480 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560a36c5-cdeb-4872-92be-dc52620ece7e admin admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:43:02.164930 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670a3bcc-db72-43da-a7a6-4d04ac62c17f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbddf971a2354bbdbb9da1d63f3abf88', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0788e6725a84e57a4174284c1845755', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:43:02.165866 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670a3bcc-db72-43da-a7a6-4d04ac62c17f service cinder] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:43:02.166307 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670a3bcc-db72-43da-a7a6-4d04ac62c17f service cinder] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:43:02.166762 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670a3bcc-db72-43da-a7a6-4d04ac62c17f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:43:02.177805 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-670a3bcc-db72-43da-a7a6-4d04ac62c17f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:43:02.179891 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-670a3bcc-db72-43da-a7a6-4d04ac62c17f service cinder] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:43:02.192839 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1164/2309] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 1 16:43:02 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 16:43:02.404559 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83271e65-7511-4a9d-bb7c-0e0cccd33c00 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:43:02.405065 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83271e65-7511-4a9d-bb7c-0e0cccd33c00 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:43:02.405176 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83271e65-7511-4a9d-bb7c-0e0cccd33c00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:43:02.428094 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-83271e65-7511-4a9d-bb7c-0e0cccd33c00 None None] MFA Rules not processed for user `5b62962ac2a14ae997fb637e80129beb`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:43:02.429651 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-83271e65-7511-4a9d-bb7c-0e0cccd33c00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:43:02.525020 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1146/2310] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 1 16:43:02 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 16:43:05.200604 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04a322d-5670-4b57-81cc-148d1cb2ffc0 None None] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:43:05.201641 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04a322d-5670-4b57-81cc-148d1cb2ffc0 None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:43:05.201731 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04a322d-5670-4b57-81cc-148d1cb2ffc0 None None] PATH_INFO: `/` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:43:05.203510 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1165/2311] 10.222.0.57 () {54 vars in 921 bytes} [Fri Jul 1 16:43:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 16:43:05.210304 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51ac5e3-ee3e-41f3-9b12-dc893c0792cb None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:43:05.210838 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51ac5e3-ee3e-41f3-9b12-dc893c0792cb None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:43:05.210937 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51ac5e3-ee3e-41f3-9b12-dc893c0792cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:43:05.247164 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-c51ac5e3-ee3e-41f3-9b12-dc893c0792cb None None] MFA Rules not processed for user `7a50f745b31941b7a7265c0fc852f48e`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:43:05.248833 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-c51ac5e3-ee3e-41f3-9b12-dc893c0792cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:43:05.348642 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1147/2312] 10.222.0.57 () {58 vars in 1018 bytes} [Fri Jul 1 16:43:05 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 16:43:05.360957 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04a322d-5670-4b57-81cc-148d1cb2ffc0 None None] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:43:05.363253 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-f04a322d-5670-4b57-81cc-148d1cb2ffc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:43:05.377970 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:43:05.459049 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04a322d-5670-4b57-81cc-148d1cb2ffc0 None None] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:43:05.468274 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8dc7cd-b003-4c45-800f-10fd82c2c3aa service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd028715bfea43099e7c6cde429ecf92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0788e6725a84e57a4174284c1845755', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:43:05.468942 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8dc7cd-b003-4c45-800f-10fd82c2c3aa service glance] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:43:05.469692 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8dc7cd-b003-4c45-800f-10fd82c2c3aa service glance] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:43:05.469692 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8dc7cd-b003-4c45-800f-10fd82c2c3aa service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:43:05.490985 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f8dc7cd-b003-4c45-800f-10fd82c2c3aa service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:43:05.492654 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f8dc7cd-b003-4c45-800f-10fd82c2c3aa service glance] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:43:05.519073 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1166/2313] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 1 16:43:05 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 16:43:05.625272 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d617656-1d23-4ec1-9e54-abe96d5a96dd None None] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:43:05.625819 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d617656-1d23-4ec1-9e54-abe96d5a96dd None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:43:05.628062 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d617656-1d23-4ec1-9e54-abe96d5a96dd None None] PATH_INFO: `/` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:43:05.629954 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1148/2314] 10.222.0.57 () {54 vars in 921 bytes} [Fri Jul 1 16:43:05 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 01 16:43:05.635598 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b548c624-8525-4cfc-933a-6f00411a0eb6 None None] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:43:05.635892 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b548c624-8525-4cfc-933a-6f00411a0eb6 None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:43:05.636203 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b548c624-8525-4cfc-933a-6f00411a0eb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:43:05.658432 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.auth.core [None req-b548c624-8525-4cfc-933a-6f00411a0eb6 None None] MFA Rules not processed for user `4ed200ce29684c5f80354fa9a8638fe2`. Rule list: `[]` (Enabled: `True`). {{(pid=80312) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:43:05.659991 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-b548c624-8525-4cfc-933a-6f00411a0eb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:43:05.770538 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1167/2315] 10.222.0.57 () {58 vars in 1018 bytes} [Fri Jul 1 16:43:05 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 16:43:05.786508 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d617656-1d23-4ec1-9e54-abe96d5a96dd None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:43:05.786508 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-4d617656-1d23-4ec1-9e54-abe96d5a96dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:43:05.799712 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d617656-1d23-4ec1-9e54-abe96d5a96dd None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:43:05.804654 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15028375-bff3-4482-86df-ecda1f69f40b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ed200ce29684c5f80354fa9a8638fe2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0788e6725a84e57a4174284c1845755', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:43:05.805308 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15028375-bff3-4482-86df-ecda1f69f40b service neutron] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:43:05.805593 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15028375-bff3-4482-86df-ecda1f69f40b service neutron] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:43:05.805871 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15028375-bff3-4482-86df-ecda1f69f40b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:43:05.814507 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15028375-bff3-4482-86df-ecda1f69f40b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:43:05.816002 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15028375-bff3-4482-86df-ecda1f69f40b service neutron] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:43:05.824337 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1149/2316] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 1 16:43:05 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 16:43:05.910273 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb1b9ad-6bc4-4a5d-9e70-4ff082219be9 None None] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:43:05.910641 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb1b9ad-6bc4-4a5d-9e70-4ff082219be9 None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:43:05.910920 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb1b9ad-6bc4-4a5d-9e70-4ff082219be9 None None] PATH_INFO: `/` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:43:05.912321 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1168/2317] 10.222.0.57 () {54 vars in 921 bytes} [Fri Jul 1 16:43:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 16:43:05.917860 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c7f360-63a9-4827-8206-b554581aa601 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:43:05.918189 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c7f360-63a9-4827-8206-b554581aa601 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:43:05.918497 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c7f360-63a9-4827-8206-b554581aa601 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:43:05.925314 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:43:05.938882 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-13c7f360-63a9-4827-8206-b554581aa601 None None] MFA Rules not processed for user `7a50f745b31941b7a7265c0fc852f48e`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:43:05.940075 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-13c7f360-63a9-4827-8206-b554581aa601 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:43:05.949240 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1150/2318] 10.222.0.57 () {58 vars in 1018 bytes} [Fri Jul 1 16:43:05 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 16:43:05.960121 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb1b9ad-6bc4-4a5d-9e70-4ff082219be9 None None] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:43:05.961800 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-deb1b9ad-6bc4-4a5d-9e70-4ff082219be9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:43:05.975242 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb1b9ad-6bc4-4a5d-9e70-4ff082219be9 None None] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:43:05.978968 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd569284-86d7-461d-9044-32d063cf4556 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ed200ce29684c5f80354fa9a8638fe2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0788e6725a84e57a4174284c1845755', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:43:05.979645 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd569284-86d7-461d-9044-32d063cf4556 service neutron] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:43:05.979929 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd569284-86d7-461d-9044-32d063cf4556 service neutron] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:43:05.980209 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd569284-86d7-461d-9044-32d063cf4556 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:43:05.991384 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd569284-86d7-461d-9044-32d063cf4556 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:43:05.992815 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd569284-86d7-461d-9044-32d063cf4556 service neutron] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:43:06.004379 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1169/2319] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 1 16:43:05 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 16:43:15.933066 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=2 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:43:15.999823 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=51 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:43:17.276710 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c385d6-d50e-4efa-b287-22c003f67062 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:43:17.277177 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c385d6-d50e-4efa-b287-22c003f67062 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:43:17.277267 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c385d6-d50e-4efa-b287-22c003f67062 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:43:17.297094 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-31c385d6-d50e-4efa-b287-22c003f67062 None None] MFA Rules not processed for user `c4de1b1eeaed4fc5ae7200b2c7a90957`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:43:17.298285 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-31c385d6-d50e-4efa-b287-22c003f67062 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:43:17.374394 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1151/2320] 10.222.0.57 () {58 vars in 1009 bytes} [Fri Jul 1 16:43:17 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 16:43:17.389534 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd569284-86d7-461d-9044-32d063cf4556 service neutron] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:43:17.391393 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-bd569284-86d7-461d-9044-32d063cf4556 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:43:17.475662 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd569284-86d7-461d-9044-32d063cf4556 service neutron] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:43:17.480639 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51497bf6-c7e0-4214-9a5a-8ea2e6923075 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '859fe410c2d2450fb803c6244839f27c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0788e6725a84e57a4174284c1845755', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:43:17.481287 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51497bf6-c7e0-4214-9a5a-8ea2e6923075 service swift] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:43:17.481605 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51497bf6-c7e0-4214-9a5a-8ea2e6923075 service swift] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:43:17.481882 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51497bf6-c7e0-4214-9a5a-8ea2e6923075 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:43:17.491094 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51497bf6-c7e0-4214-9a5a-8ea2e6923075 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:43:17.492688 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51497bf6-c7e0-4214-9a5a-8ea2e6923075 service swift] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:43:17.501989 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1170/2321] 10.222.0.57 () {60 vars in 1384 bytes} [Fri Jul 1 16:43:17 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 01 16:43:20.068193 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7e2447-321a-44ce-924d-ea4bc7c8cb40 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:43:20.068812 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7e2447-321a-44ce-924d-ea4bc7c8cb40 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:43:20.068812 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7e2447-321a-44ce-924d-ea4bc7c8cb40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:43:20.087738 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-2a7e2447-321a-44ce-924d-ea4bc7c8cb40 None None] MFA Rules not processed for user `c4de1b1eeaed4fc5ae7200b2c7a90957`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:43:20.089317 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-2a7e2447-321a-44ce-924d-ea4bc7c8cb40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:43:20.098164 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1152/2322] 10.222.0.57 () {58 vars in 1009 bytes} [Fri Jul 1 16:43:20 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 16:43:20.123630 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51497bf6-c7e0-4214-9a5a-8ea2e6923075 service swift] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:43:20.137063 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51497bf6-c7e0-4214-9a5a-8ea2e6923075 service swift] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:43:20.143173 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2be4f0-5851-4cde-9ffb-e32fe9715a21 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '859fe410c2d2450fb803c6244839f27c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0788e6725a84e57a4174284c1845755', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:43:20.143844 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2be4f0-5851-4cde-9ffb-e32fe9715a21 service swift] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:43:20.144130 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2be4f0-5851-4cde-9ffb-e32fe9715a21 service swift] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:43:20.144408 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2be4f0-5851-4cde-9ffb-e32fe9715a21 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:43:20.154485 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a2be4f0-5851-4cde-9ffb-e32fe9715a21 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:43:20.156170 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a2be4f0-5851-4cde-9ffb-e32fe9715a21 service swift] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:43:20.166962 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1171/2323] 10.222.0.57 () {60 vars in 1384 bytes} [Fri Jul 1 16:43:20 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 01 16:43:30.091495 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=31 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:43:30.161257 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=38 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:43:32.501028 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7e2447-321a-44ce-924d-ea4bc7c8cb40 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:43:32.503563 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-2a7e2447-321a-44ce-924d-ea4bc7c8cb40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:43:32.607047 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7e2447-321a-44ce-924d-ea4bc7c8cb40 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:43:32.612069 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea0f429-5c1c-48ab-857a-060a2136b434 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc38af18dec4e9b8143df23af6aad43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0788e6725a84e57a4174284c1845755', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:43:32.612753 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea0f429-5c1c-48ab-857a-060a2136b434 service placement] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:43:32.613087 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea0f429-5c1c-48ab-857a-060a2136b434 service placement] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:43:32.613678 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea0f429-5c1c-48ab-857a-060a2136b434 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:43:32.616446 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-fea0f429-5c1c-48ab-857a-060a2136b434 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:43:32.630022 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea0f429-5c1c-48ab-857a-060a2136b434 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:43:32.631657 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea0f429-5c1c-48ab-857a-060a2136b434 service placement] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:43:32.644056 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1153/2324] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 1 16:43:32 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 16:43:42.639866 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=41 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:43:43.259150 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd48820-10fe-4b7b-b312-576c1da96b62 None None] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:43:43.259688 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd48820-10fe-4b7b-b312-576c1da96b62 None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:43:43.259791 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd48820-10fe-4b7b-b312-576c1da96b62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:43:43.320413 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.auth.core [None req-7cd48820-10fe-4b7b-b312-576c1da96b62 None None] MFA Rules not processed for user `c4de1b1eeaed4fc5ae7200b2c7a90957`. Rule list: `[]` (Enabled: `True`). {{(pid=80312) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:43:43.322969 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-7cd48820-10fe-4b7b-b312-576c1da96b62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:43:43.350009 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1172/2325] 10.222.0.57 () {58 vars in 967 bytes} [Fri Jul 1 16:43:43 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 16:43:43.366838 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea0f429-5c1c-48ab-857a-060a2136b434 service placement] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:43:43.394779 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea0f429-5c1c-48ab-857a-060a2136b434 service placement] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:43:43.400395 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9fe4df-6ae8-4652-9521-0e76edaed266 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '859fe410c2d2450fb803c6244839f27c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0788e6725a84e57a4174284c1845755', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:43:43.400981 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9fe4df-6ae8-4652-9521-0e76edaed266 service swift] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:43:43.401266 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9fe4df-6ae8-4652-9521-0e76edaed266 service swift] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:43:43.401541 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9fe4df-6ae8-4652-9521-0e76edaed266 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:43:43.413545 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a9fe4df-6ae8-4652-9521-0e76edaed266 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:43:43.415204 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a9fe4df-6ae8-4652-9521-0e76edaed266 service swift] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:43:43.426101 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1154/2326] 10.222.0.57 () {60 vars in 1384 bytes} [Fri Jul 1 16:43:43 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 01 16:43:45.533701 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd48820-10fe-4b7b-b312-576c1da96b62 None None] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:43:45.535200 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-7cd48820-10fe-4b7b-b312-576c1da96b62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:43:45.622571 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd48820-10fe-4b7b-b312-576c1da96b62 None None] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:43:45.629014 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431940aa-1546-4c2b-80f2-15a8076e2009 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:43:45.630452 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431940aa-1546-4c2b-80f2-15a8076e2009 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:43:45.630743 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431940aa-1546-4c2b-80f2-15a8076e2009 admin admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:43:45.631441 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431940aa-1546-4c2b-80f2-15a8076e2009 admin admin] PATH_INFO: `/v3/users/1f2a1540974a444f8341d395d6a2b070` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:43:45.633652 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431940aa-1546-4c2b-80f2-15a8076e2009 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1f2a1540974a444f8341d395d6a2b070)` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:43:45.635833 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431940aa-1546-4c2b-80f2-15a8076e2009 admin admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:43:45.701868 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-431940aa-1546-4c2b-80f2-15a8076e2009 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1f2a1540974a444f8341d395d6a2b070'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:43:45.719809 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-431940aa-1546-4c2b-80f2-15a8076e2009 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1f2a1540974a444f8341d395d6a2b070'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:43:45.727411 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-431940aa-1546-4c2b-80f2-15a8076e2009 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1f2a1540974a444f8341d395d6a2b070'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:43:45.728269 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-431940aa-1546-4c2b-80f2-15a8076e2009 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1f2a1540974a444f8341d395d6a2b070'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:43:45.730787 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-431940aa-1546-4c2b-80f2-15a8076e2009 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1f2a1540974a444f8341d395d6a2b070'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:43:45.748964 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1173/2327] 10.222.0.57 () {58 vars in 1223 bytes} [Fri Jul 1 16:43:45 2022] DELETE /identity/v3/users/1f2a1540974a444f8341d395d6a2b070 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 16:43:45.937795 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9fe4df-6ae8-4652-9521-0e76edaed266 service swift] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:43:45.950173 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9fe4df-6ae8-4652-9521-0e76edaed266 service swift] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:43:46.013174 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d2f2e7-41de-45c7-9867-56e545ae17b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:43:46.014059 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d2f2e7-41de-45c7-9867-56e545ae17b1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:43:46.014339 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d2f2e7-41de-45c7-9867-56e545ae17b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:43:46.014646 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d2f2e7-41de-45c7-9867-56e545ae17b1 admin admin] PATH_INFO: `/v3/projects/b6598cf49d3a4426be411fa45be0c7c8` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:43:46.016120 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d2f2e7-41de-45c7-9867-56e545ae17b1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b6598cf49d3a4426be411fa45be0c7c8)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:43:46.018083 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d2f2e7-41de-45c7-9867-56e545ae17b1 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:43:46.072721 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-f2d2f2e7-41de-45c7-9867-56e545ae17b1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b6598cf49d3a4426be411fa45be0c7c8', 'request_id': 'req-f2d2f2e7-41de-45c7-9867-56e545ae17b1'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:43:46.078988 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-f2d2f2e7-41de-45c7-9867-56e545ae17b1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b6598cf49d3a4426be411fa45be0c7c8', 'request_id': 'req-f2d2f2e7-41de-45c7-9867-56e545ae17b1'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:43:46.085980 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-f2d2f2e7-41de-45c7-9867-56e545ae17b1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b6598cf49d3a4426be411fa45be0c7c8', 'request_id': 'req-f2d2f2e7-41de-45c7-9867-56e545ae17b1'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:43:46.099229 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-f2d2f2e7-41de-45c7-9867-56e545ae17b1 admin admin] The token cache is being invalidate because project b6598cf49d3a4426be411fa45be0c7c8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80311) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 01 16:43:46.099522 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-f2d2f2e7-41de-45c7-9867-56e545ae17b1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:43:46.100146 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-f2d2f2e7-41de-45c7-9867-56e545ae17b1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:43:46.101753 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1155/2328] 10.222.0.57 () {58 vars in 1232 bytes} [Fri Jul 1 16:43:45 2022] DELETE /identity/v3/projects/b6598cf49d3a4426be411fa45be0c7c8 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 16:43:46.106860 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a95169c-8c98-43f3-b446-2479a4dc8d71 None None] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:43:46.107235 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a95169c-8c98-43f3-b446-2479a4dc8d71 None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:43:46.107702 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a95169c-8c98-43f3-b446-2479a4dc8d71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:43:46.127606 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.auth.core [None req-2a95169c-8c98-43f3-b446-2479a4dc8d71 None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80312) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:43:46.128814 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-2a95169c-8c98-43f3-b446-2479a4dc8d71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:43:46.163375 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:43:46.194569 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1174/2329] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 1 16:43:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 16:43:46.202565 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d9735d-7fc1-4398-a43c-3e2fd634c141 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:43:46.203246 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d9735d-7fc1-4398-a43c-3e2fd634c141 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:43:46.203532 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d9735d-7fc1-4398-a43c-3e2fd634c141 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:43:46.223575 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-08d9735d-7fc1-4398-a43c-3e2fd634c141 None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:43:46.224790 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-08d9735d-7fc1-4398-a43c-3e2fd634c141 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:43:46.236440 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1156/2330] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 1 16:43:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 16:43:46.238859 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a95169c-8c98-43f3-b446-2479a4dc8d71 None None] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:43:46.249529 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a95169c-8c98-43f3-b446-2479a4dc8d71 None None] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:43:46.256801 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7815f248-0471-465a-99c6-a8840f59a417 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:43:46.257420 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7815f248-0471-465a-99c6-a8840f59a417 admin admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:43:46.257689 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7815f248-0471-465a-99c6-a8840f59a417 admin admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:43:46.257963 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7815f248-0471-465a-99c6-a8840f59a417 admin admin] PATH_INFO: `/v3/domains` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:43:46.258743 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7815f248-0471-465a-99c6-a8840f59a417 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80312) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:43:46.259054 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7815f248-0471-465a-99c6-a8840f59a417 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:43:46.260698 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7815f248-0471-465a-99c6-a8840f59a417 admin admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:43:46.276566 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1175/2331] 10.222.0.57 () {58 vars in 1152 bytes} [Fri Jul 1 16:43:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:43:46.280843 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d9735d-7fc1-4398-a43c-3e2fd634c141 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:43:46.296206 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d9735d-7fc1-4398-a43c-3e2fd634c141 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:43:46.301341 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3497d1-5be5-4e8c-ae9a-33573c1e088a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:43:46.302122 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3497d1-5be5-4e8c-ae9a-33573c1e088a admin admin] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:43:46.302542 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3497d1-5be5-4e8c-ae9a-33573c1e088a admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:43:46.302919 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3497d1-5be5-4e8c-ae9a-33573c1e088a admin admin] PATH_INFO: `/v3/projects` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:43:46.303728 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad3497d1-5be5-4e8c-ae9a-33573c1e088a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:43:46.305908 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad3497d1-5be5-4e8c-ae9a-33573c1e088a admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:43:46.342967 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1157/2332] 10.222.0.57 () {60 vars in 1152 bytes} [Fri Jul 1 16:43:46 2022] POST /identity/v3/projects => generated 369 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 16:43:46.344435 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7815f248-0471-465a-99c6-a8840f59a417 admin admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:43:46.356257 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7815f248-0471-465a-99c6-a8840f59a417 admin admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:43:46.420183 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5644378d-b7c3-4192-928d-163bf871f3f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:43:46.420864 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5644378d-b7c3-4192-928d-163bf871f3f3 admin admin] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:43:46.421150 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5644378d-b7c3-4192-928d-163bf871f3f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:43:46.421429 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5644378d-b7c3-4192-928d-163bf871f3f3 admin admin] PATH_INFO: `/v3/users` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:43:46.422122 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5644378d-b7c3-4192-928d-163bf871f3f3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:43:46.424725 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5644378d-b7c3-4192-928d-163bf871f3f3 admin admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:43:46.445998 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1176/2333] 10.222.0.57 () {60 vars in 1143 bytes} [Fri Jul 1 16:43:46 2022] POST /identity/v3/users => generated 285 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 01 16:43:46.450967 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3497d1-5be5-4e8c-ae9a-33573c1e088a admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:43:46.461721 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3497d1-5be5-4e8c-ae9a-33573c1e088a admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:43:46.467453 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116374f3-035c-407e-be92-ca3317c8a2ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:43:46.468130 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116374f3-035c-407e-be92-ca3317c8a2ed admin admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:43:46.468420 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116374f3-035c-407e-be92-ca3317c8a2ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:43:46.468725 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116374f3-035c-407e-be92-ca3317c8a2ed admin admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:43:46.469446 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-116374f3-035c-407e-be92-ca3317c8a2ed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:43:46.471101 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-116374f3-035c-407e-be92-ca3317c8a2ed admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:43:46.482383 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1158/2334] 10.222.0.57 () {58 vars in 1121 bytes} [Fri Jul 1 16:43:46 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 16:43:46.486779 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5644378d-b7c3-4192-928d-163bf871f3f3 admin admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:43:46.497675 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5644378d-b7c3-4192-928d-163bf871f3f3 admin admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:43:46.502624 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd61c7e-685f-44a4-85f9-0376d87c247a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:43:46.503296 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd61c7e-685f-44a4-85f9-0376d87c247a admin admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:43:46.503612 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd61c7e-685f-44a4-85f9-0376d87c247a admin admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:43:46.503893 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd61c7e-685f-44a4-85f9-0376d87c247a admin admin] PATH_INFO: `/v3/roles` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:43:46.504631 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd61c7e-685f-44a4-85f9-0376d87c247a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:43:46.506243 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd61c7e-685f-44a4-85f9-0376d87c247a admin admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:43:46.517721 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1177/2335] 10.222.0.57 () {58 vars in 1121 bytes} [Fri Jul 1 16:43:46 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 16:43:46.521876 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116374f3-035c-407e-be92-ca3317c8a2ed admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:43:46.532180 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116374f3-035c-407e-be92-ca3317c8a2ed admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:43:46.536843 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d92a43b-b8a4-4380-8a1c-8d33beab3584 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:43:46.537728 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d92a43b-b8a4-4380-8a1c-8d33beab3584 admin admin] REQUEST_METHOD: `PUT` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:43:46.538013 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d92a43b-b8a4-4380-8a1c-8d33beab3584 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:43:46.538313 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d92a43b-b8a4-4380-8a1c-8d33beab3584 admin admin] PATH_INFO: `/v3/projects/a5859bbd52284320951b202559ea2e51/users/854a2f9474a54c229e524d5da7537118/roles/983be40bdab04c478eaa90c6b6a46b14` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:43:46.553898 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d92a43b-b8a4-4380-8a1c-8d33beab3584 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a5859bbd52284320951b202559ea2e51, user_id=854a2f9474a54c229e524d5da7537118, role_id=983be40bdab04c478eaa90c6b6a46b14)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:43:46.556604 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d92a43b-b8a4-4380-8a1c-8d33beab3584 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:43:46.570980 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1159/2336] 10.222.0.57 () {60 vars in 1482 bytes} [Fri Jul 1 16:43:46 2022] PUT /identity/v3/projects/a5859bbd52284320951b202559ea2e51/users/854a2f9474a54c229e524d5da7537118/roles/983be40bdab04c478eaa90c6b6a46b14 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 16:43:46.581874 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f37d2ec-4e0a-4a12-8857-7712e4293d9f None None] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:43:46.582389 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f37d2ec-4e0a-4a12-8857-7712e4293d9f None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:43:46.582477 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f37d2ec-4e0a-4a12-8857-7712e4293d9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:43:46.609921 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.auth.core [None req-2f37d2ec-4e0a-4a12-8857-7712e4293d9f None None] MFA Rules not processed for user `854a2f9474a54c229e524d5da7537118`. Rule list: `[]` (Enabled: `True`). {{(pid=80312) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:43:46.611186 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-2f37d2ec-4e0a-4a12-8857-7712e4293d9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:43:46.675639 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1178/2337] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 1 16:43:46 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 16:43:46.691574 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d92a43b-b8a4-4380-8a1c-8d33beab3584 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:43:46.693386 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-3d92a43b-b8a4-4380-8a1c-8d33beab3584 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:43:46.699821 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:43:46.782546 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d92a43b-b8a4-4380-8a1c-8d33beab3584 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:43:46.786746 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b0ca8f-04ff-409e-b3ff-5a72c19f4230 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd028715bfea43099e7c6cde429ecf92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0788e6725a84e57a4174284c1845755', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:43:46.787397 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b0ca8f-04ff-409e-b3ff-5a72c19f4230 service glance] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:43:46.787682 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b0ca8f-04ff-409e-b3ff-5a72c19f4230 service glance] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:43:46.787958 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b0ca8f-04ff-409e-b3ff-5a72c19f4230 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:43:46.798891 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b0ca8f-04ff-409e-b3ff-5a72c19f4230 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:43:46.800332 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b0ca8f-04ff-409e-b3ff-5a72c19f4230 service glance] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:43:46.809779 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1160/2338] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 1 16:43:46 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 16:43:48.777845 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e60ca0-1c7c-4289-84af-71ad061668d9 None None] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:43:48.778441 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e60ca0-1c7c-4289-84af-71ad061668d9 None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:43:48.778441 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e60ca0-1c7c-4289-84af-71ad061668d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:43:48.797155 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.auth.core [None req-30e60ca0-1c7c-4289-84af-71ad061668d9 None None] MFA Rules not processed for user `c4de1b1eeaed4fc5ae7200b2c7a90957`. Rule list: `[]` (Enabled: `True`). {{(pid=80312) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:43:48.798401 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-30e60ca0-1c7c-4289-84af-71ad061668d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:43:48.837037 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:43:48.865164 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1179/2339] 10.222.0.57 () {58 vars in 967 bytes} [Fri Jul 1 16:43:48 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 16:43:48.877546 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b0ca8f-04ff-409e-b3ff-5a72c19f4230 service glance] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:43:48.879082 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-16b0ca8f-04ff-409e-b3ff-5a72c19f4230 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:43:48.971289 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b0ca8f-04ff-409e-b3ff-5a72c19f4230 service glance] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:43:48.976487 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e88ea99-8bfd-4b5f-8483-4af4bf45d569 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '859fe410c2d2450fb803c6244839f27c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0788e6725a84e57a4174284c1845755', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:43:48.977144 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e88ea99-8bfd-4b5f-8483-4af4bf45d569 service swift] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:43:48.977431 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e88ea99-8bfd-4b5f-8483-4af4bf45d569 service swift] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:43:48.977713 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e88ea99-8bfd-4b5f-8483-4af4bf45d569 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:43:48.986628 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e88ea99-8bfd-4b5f-8483-4af4bf45d569 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:43:48.988252 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e88ea99-8bfd-4b5f-8483-4af4bf45d569 service swift] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:43:48.997232 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1161/2340] 10.222.0.57 () {60 vars in 1384 bytes} [Fri Jul 1 16:43:48 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 01 16:43:57.460434 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e60ca0-1c7c-4289-84af-71ad061668d9 None None] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:43:57.461919 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-30e60ca0-1c7c-4289-84af-71ad061668d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:43:57.544619 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e60ca0-1c7c-4289-84af-71ad061668d9 None None] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:43:57.549096 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3befb21-1844-47c4-b858-847f257e70f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:43:57.549960 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3befb21-1844-47c4-b858-847f257e70f0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:43:57.550243 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3befb21-1844-47c4-b858-847f257e70f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:43:57.550553 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3befb21-1844-47c4-b858-847f257e70f0 admin admin] PATH_INFO: `/v3/users/87ce0bd7ff6f4e7a9a0b1c9b869f31c5` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:43:57.551854 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3befb21-1844-47c4-b858-847f257e70f0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=87ce0bd7ff6f4e7a9a0b1c9b869f31c5)` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:43:57.553764 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3befb21-1844-47c4-b858-847f257e70f0 admin admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:43:57.601414 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-f3befb21-1844-47c4-b858-847f257e70f0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '87ce0bd7ff6f4e7a9a0b1c9b869f31c5'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:43:57.614173 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-f3befb21-1844-47c4-b858-847f257e70f0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '87ce0bd7ff6f4e7a9a0b1c9b869f31c5'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:43:57.622495 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-f3befb21-1844-47c4-b858-847f257e70f0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '87ce0bd7ff6f4e7a9a0b1c9b869f31c5'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:43:57.623204 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-f3befb21-1844-47c4-b858-847f257e70f0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '87ce0bd7ff6f4e7a9a0b1c9b869f31c5'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:43:57.623800 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-f3befb21-1844-47c4-b858-847f257e70f0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '87ce0bd7ff6f4e7a9a0b1c9b869f31c5'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:43:57.635762 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1180/2341] 10.222.0.57 () {58 vars in 1223 bytes} [Fri Jul 1 16:43:57 2022] DELETE /identity/v3/users/87ce0bd7ff6f4e7a9a0b1c9b869f31c5 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 16:43:57.640392 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d1e0a9-11b7-460f-b528-85cd144f7dca None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:43:57.640853 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d1e0a9-11b7-460f-b528-85cd144f7dca None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:43:57.640966 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d1e0a9-11b7-460f-b528-85cd144f7dca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:43:57.662264 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-c4d1e0a9-11b7-460f-b528-85cd144f7dca None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:43:57.663724 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-c4d1e0a9-11b7-460f-b528-85cd144f7dca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:43:57.732077 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1162/2342] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 1 16:43:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 16:43:57.741731 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3befb21-1844-47c4-b858-847f257e70f0 admin admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:43:57.743207 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-f3befb21-1844-47c4-b858-847f257e70f0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:43:57.812971 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3befb21-1844-47c4-b858-847f257e70f0 admin admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:43:57.817355 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3fa6511-3d82-4009-b59b-f66d52306e04 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ed200ce29684c5f80354fa9a8638fe2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0788e6725a84e57a4174284c1845755', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:43:57.818653 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fa6511-3d82-4009-b59b-f66d52306e04 service neutron] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:43:57.818983 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fa6511-3d82-4009-b59b-f66d52306e04 service neutron] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:43:57.819370 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fa6511-3d82-4009-b59b-f66d52306e04 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:43:57.829723 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3fa6511-3d82-4009-b59b-f66d52306e04 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:43:57.831332 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3fa6511-3d82-4009-b59b-f66d52306e04 service neutron] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:43:57.844340 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1181/2343] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 1 16:43:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 16:43:58.135012 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d1e0a9-11b7-460f-b528-85cd144f7dca None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:43:58.138852 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-c4d1e0a9-11b7-460f-b528-85cd144f7dca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:43:58.161164 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d1e0a9-11b7-460f-b528-85cd144f7dca None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:43:58.169584 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac58175-b9bb-4f77-a75f-a100276f6ac9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:43:58.171244 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac58175-b9bb-4f77-a75f-a100276f6ac9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:43:58.171833 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac58175-b9bb-4f77-a75f-a100276f6ac9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:43:58.172419 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac58175-b9bb-4f77-a75f-a100276f6ac9 admin admin] PATH_INFO: `/v3/projects/426eb4d841c2477295ef705b1eba6d48` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:43:58.174671 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac58175-b9bb-4f77-a75f-a100276f6ac9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=426eb4d841c2477295ef705b1eba6d48)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:43:58.177224 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac58175-b9bb-4f77-a75f-a100276f6ac9 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:43:58.179447 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:43:58.229325 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-bac58175-b9bb-4f77-a75f-a100276f6ac9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '426eb4d841c2477295ef705b1eba6d48', 'request_id': 'req-bac58175-b9bb-4f77-a75f-a100276f6ac9'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:43:58.236280 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-bac58175-b9bb-4f77-a75f-a100276f6ac9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '426eb4d841c2477295ef705b1eba6d48', 'request_id': 'req-bac58175-b9bb-4f77-a75f-a100276f6ac9'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:43:58.242527 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-bac58175-b9bb-4f77-a75f-a100276f6ac9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '426eb4d841c2477295ef705b1eba6d48', 'request_id': 'req-bac58175-b9bb-4f77-a75f-a100276f6ac9'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:43:58.256341 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-bac58175-b9bb-4f77-a75f-a100276f6ac9 admin admin] The token cache is being invalidate because project 426eb4d841c2477295ef705b1eba6d48 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80311) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 01 16:43:58.256547 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-bac58175-b9bb-4f77-a75f-a100276f6ac9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:43:58.257215 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-bac58175-b9bb-4f77-a75f-a100276f6ac9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:43:58.259109 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1163/2344] 10.222.0.57 () {58 vars in 1232 bytes} [Fri Jul 1 16:43:58 2022] DELETE /identity/v3/projects/426eb4d841c2477295ef705b1eba6d48 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 16:43:58.264699 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6f1e9c-4bae-4f69-8862-769d3ec2594f None None] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:43:58.264993 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6f1e9c-4bae-4f69-8862-769d3ec2594f None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:43:58.265305 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6f1e9c-4bae-4f69-8862-769d3ec2594f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:43:58.271692 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:43:58.285910 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.auth.core [None req-3d6f1e9c-4bae-4f69-8862-769d3ec2594f None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80312) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:43:58.287147 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-3d6f1e9c-4bae-4f69-8862-769d3ec2594f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:43:58.347626 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1182/2345] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 1 16:43:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 16:43:58.356325 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d6dd50-bfb3-4733-9b62-5a58ab61fd57 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:43:58.356660 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d6dd50-bfb3-4733-9b62-5a58ab61fd57 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:43:58.357032 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d6dd50-bfb3-4733-9b62-5a58ab61fd57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:43:58.375949 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-02d6dd50-bfb3-4733-9b62-5a58ab61fd57 None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:43:58.377151 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-02d6dd50-bfb3-4733-9b62-5a58ab61fd57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:43:58.385003 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1164/2346] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 1 16:43:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 16:43:58.389281 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6f1e9c-4bae-4f69-8862-769d3ec2594f None None] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:43:58.399202 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6f1e9c-4bae-4f69-8862-769d3ec2594f None None] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:43:58.404453 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e105b59e-9dc7-4aa2-a0a4-7b7ebf7dfda1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:43:58.405109 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e105b59e-9dc7-4aa2-a0a4-7b7ebf7dfda1 admin admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:43:58.405394 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e105b59e-9dc7-4aa2-a0a4-7b7ebf7dfda1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:43:58.405672 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e105b59e-9dc7-4aa2-a0a4-7b7ebf7dfda1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:43:58.406625 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e105b59e-9dc7-4aa2-a0a4-7b7ebf7dfda1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80312) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:43:58.406987 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e105b59e-9dc7-4aa2-a0a4-7b7ebf7dfda1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:43:58.408644 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e105b59e-9dc7-4aa2-a0a4-7b7ebf7dfda1 admin admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:43:58.421420 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1183/2347] 10.222.0.57 () {58 vars in 1152 bytes} [Fri Jul 1 16:43:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:43:58.425633 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d6dd50-bfb3-4733-9b62-5a58ab61fd57 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:43:58.437028 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d6dd50-bfb3-4733-9b62-5a58ab61fd57 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:43:58.441880 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae9535f-26aa-42c9-9026-da5420737141 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:43:58.442804 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae9535f-26aa-42c9-9026-da5420737141 admin admin] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:43:58.443128 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae9535f-26aa-42c9-9026-da5420737141 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:43:58.443405 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae9535f-26aa-42c9-9026-da5420737141 admin admin] PATH_INFO: `/v3/projects` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:43:58.444115 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae9535f-26aa-42c9-9026-da5420737141 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:43:58.446254 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae9535f-26aa-42c9-9026-da5420737141 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:43:58.478933 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1165/2348] 10.222.0.57 () {60 vars in 1152 bytes} [Fri Jul 1 16:43:58 2022] POST /identity/v3/projects => generated 391 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 16:43:58.480625 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e105b59e-9dc7-4aa2-a0a4-7b7ebf7dfda1 admin admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:43:58.491170 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e105b59e-9dc7-4aa2-a0a4-7b7ebf7dfda1 admin admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:43:58.556957 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19471783-b331-4d8e-8a35-20f0b9ed5df9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:43:58.557631 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19471783-b331-4d8e-8a35-20f0b9ed5df9 admin admin] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:43:58.557965 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19471783-b331-4d8e-8a35-20f0b9ed5df9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:43:58.558879 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19471783-b331-4d8e-8a35-20f0b9ed5df9 admin admin] PATH_INFO: `/v3/users` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:43:58.559614 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19471783-b331-4d8e-8a35-20f0b9ed5df9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:43:58.562076 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19471783-b331-4d8e-8a35-20f0b9ed5df9 admin admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:43:58.590291 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1184/2349] 10.222.0.57 () {60 vars in 1143 bytes} [Fri Jul 1 16:43:58 2022] POST /identity/v3/users => generated 296 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 16:43:58.591806 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae9535f-26aa-42c9-9026-da5420737141 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:43:58.603668 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae9535f-26aa-42c9-9026-da5420737141 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:43:58.609238 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698e9e27-f9ac-40f8-96be-6a41c6409091 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:43:58.609945 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698e9e27-f9ac-40f8-96be-6a41c6409091 admin admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:43:58.610234 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698e9e27-f9ac-40f8-96be-6a41c6409091 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:43:58.610558 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698e9e27-f9ac-40f8-96be-6a41c6409091 admin admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:43:58.611303 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-698e9e27-f9ac-40f8-96be-6a41c6409091 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:43:58.612948 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-698e9e27-f9ac-40f8-96be-6a41c6409091 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:43:58.625473 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1166/2350] 10.222.0.57 () {58 vars in 1121 bytes} [Fri Jul 1 16:43:58 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 16:43:58.629520 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19471783-b331-4d8e-8a35-20f0b9ed5df9 admin admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:43:58.641018 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19471783-b331-4d8e-8a35-20f0b9ed5df9 admin admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:43:58.646082 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7812960-93c4-4c87-b9f6-fb1ff4c5f372 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:43:58.646791 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7812960-93c4-4c87-b9f6-fb1ff4c5f372 admin admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:43:58.647091 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7812960-93c4-4c87-b9f6-fb1ff4c5f372 admin admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:43:58.647356 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7812960-93c4-4c87-b9f6-fb1ff4c5f372 admin admin] PATH_INFO: `/v3/roles` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:43:58.648100 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7812960-93c4-4c87-b9f6-fb1ff4c5f372 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:43:58.649739 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7812960-93c4-4c87-b9f6-fb1ff4c5f372 admin admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:43:58.660627 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1185/2351] 10.222.0.57 () {58 vars in 1121 bytes} [Fri Jul 1 16:43:58 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 16:43:58.664751 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698e9e27-f9ac-40f8-96be-6a41c6409091 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:43:58.676503 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698e9e27-f9ac-40f8-96be-6a41c6409091 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:43:58.682767 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46578c21-0b83-48a0-814d-d7b7baa990eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:43:58.684471 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46578c21-0b83-48a0-814d-d7b7baa990eb admin admin] REQUEST_METHOD: `PUT` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:43:58.684761 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46578c21-0b83-48a0-814d-d7b7baa990eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:43:58.685040 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46578c21-0b83-48a0-814d-d7b7baa990eb admin admin] PATH_INFO: `/v3/projects/a2ac8b6f02c7469bb7d60dfe07e4ee9c/users/a216de9eb61a4a4083620187f5289e19/roles/983be40bdab04c478eaa90c6b6a46b14` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:43:58.701518 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46578c21-0b83-48a0-814d-d7b7baa990eb admin admin] RBAC: Authorizing `identity:create_grant(project_id=a2ac8b6f02c7469bb7d60dfe07e4ee9c, user_id=a216de9eb61a4a4083620187f5289e19, role_id=983be40bdab04c478eaa90c6b6a46b14)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:43:58.704404 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46578c21-0b83-48a0-814d-d7b7baa990eb admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:43:58.735914 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1167/2352] 10.222.0.57 () {60 vars in 1482 bytes} [Fri Jul 1 16:43:58 2022] PUT /identity/v3/projects/a2ac8b6f02c7469bb7d60dfe07e4ee9c/users/a216de9eb61a4a4083620187f5289e19/roles/983be40bdab04c478eaa90c6b6a46b14 => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 16:43:58.810014 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1ac2bf-1455-4b8a-b743-b8f1100986ba None None] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:43:58.810618 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1ac2bf-1455-4b8a-b743-b8f1100986ba None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:43:58.811193 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1ac2bf-1455-4b8a-b743-b8f1100986ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:43:58.841929 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.auth.core [None req-1f1ac2bf-1455-4b8a-b743-b8f1100986ba None None] MFA Rules not processed for user `a216de9eb61a4a4083620187f5289e19`. Rule list: `[]` (Enabled: `True`). {{(pid=80312) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:43:58.843374 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-1f1ac2bf-1455-4b8a-b743-b8f1100986ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:43:58.921672 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:43:58.936678 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1186/2353] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 1 16:43:58 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 16:43:58.952992 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46578c21-0b83-48a0-814d-d7b7baa990eb admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:43:58.956348 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-46578c21-0b83-48a0-814d-d7b7baa990eb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:43:58.994030 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1ac2bf-1455-4b8a-b743-b8f1100986ba None None] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:43:58.996251 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-1f1ac2bf-1455-4b8a-b743-b8f1100986ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:43:59.040872 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46578c21-0b83-48a0-814d-d7b7baa990eb admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:43:59.045030 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668e85c5-e832-42a6-9aef-8720d7ccff3f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbddf971a2354bbdbb9da1d63f3abf88', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0788e6725a84e57a4174284c1845755', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:43:59.045677 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668e85c5-e832-42a6-9aef-8720d7ccff3f service cinder] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:43:59.045960 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668e85c5-e832-42a6-9aef-8720d7ccff3f service cinder] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:43:59.046234 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668e85c5-e832-42a6-9aef-8720d7ccff3f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:43:59.054440 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1ac2bf-1455-4b8a-b743-b8f1100986ba None None] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:43:59.054600 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-668e85c5-e832-42a6-9aef-8720d7ccff3f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:43:59.056023 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-668e85c5-e832-42a6-9aef-8720d7ccff3f service cinder] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:43:59.058780 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f771f2-5c78-495f-9dd4-97c34295cd53 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ed200ce29684c5f80354fa9a8638fe2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0788e6725a84e57a4174284c1845755', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:43:59.059444 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f771f2-5c78-495f-9dd4-97c34295cd53 service neutron] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:43:59.059731 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f771f2-5c78-495f-9dd4-97c34295cd53 service neutron] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:43:59.060011 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f771f2-5c78-495f-9dd4-97c34295cd53 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:43:59.061858 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-77f771f2-5c78-495f-9dd4-97c34295cd53 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:43:59.065869 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1168/2354] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 1 16:43:58 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 16:43:59.131692 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f771f2-5c78-495f-9dd4-97c34295cd53 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:43:59.133931 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f771f2-5c78-495f-9dd4-97c34295cd53 service neutron] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:43:59.144254 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1187/2355] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 1 16:43:58 2022] GET /identity/v3/auth/tokens => generated 3303 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 01 16:44:09.061723 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=86,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:44:09.140429 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=51 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:44:12.485159 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc27df8-fbb1-49a0-a007-21aec094b8b8 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:44:12.485661 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc27df8-fbb1-49a0-a007-21aec094b8b8 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:44:12.485756 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc27df8-fbb1-49a0-a007-21aec094b8b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:44:12.503495 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-9cc27df8-fbb1-49a0-a007-21aec094b8b8 None None] MFA Rules not processed for user `7c4da21fbfa3472cac6e766a2f4799d8`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:44:12.504703 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-9cc27df8-fbb1-49a0-a007-21aec094b8b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:44:12.561862 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1169/2356] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 1 16:44:12 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 16:44:21.580335 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f771f2-5c78-495f-9dd4-97c34295cd53 service neutron] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:44:21.651688 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f771f2-5c78-495f-9dd4-97c34295cd53 service neutron] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:44:21.656379 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441ce5f2-0b32-4e57-8a59-2f79b89321c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:44:21.657264 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441ce5f2-0b32-4e57-8a59-2f79b89321c4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:44:21.657545 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441ce5f2-0b32-4e57-8a59-2f79b89321c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:44:21.657812 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441ce5f2-0b32-4e57-8a59-2f79b89321c4 admin admin] PATH_INFO: `/v3/users/a216de9eb61a4a4083620187f5289e19` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:44:21.659274 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-441ce5f2-0b32-4e57-8a59-2f79b89321c4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a216de9eb61a4a4083620187f5289e19)` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:44:21.661410 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-441ce5f2-0b32-4e57-8a59-2f79b89321c4 admin admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:44:21.719582 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-441ce5f2-0b32-4e57-8a59-2f79b89321c4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a216de9eb61a4a4083620187f5289e19'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:44:21.733676 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-441ce5f2-0b32-4e57-8a59-2f79b89321c4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a216de9eb61a4a4083620187f5289e19'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:44:21.742304 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-441ce5f2-0b32-4e57-8a59-2f79b89321c4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a216de9eb61a4a4083620187f5289e19'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:44:21.743201 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-441ce5f2-0b32-4e57-8a59-2f79b89321c4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a216de9eb61a4a4083620187f5289e19'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:44:21.743800 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-441ce5f2-0b32-4e57-8a59-2f79b89321c4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a216de9eb61a4a4083620187f5289e19'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:44:21.765021 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1188/2357] 10.222.0.57 () {58 vars in 1223 bytes} [Fri Jul 1 16:44:21 2022] DELETE /identity/v3/users/a216de9eb61a4a4083620187f5289e19 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 16:44:21.769708 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6bb0b96-9116-4a09-ba5f-dbe195db1499 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:44:21.770179 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6bb0b96-9116-4a09-ba5f-dbe195db1499 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:44:21.770298 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6bb0b96-9116-4a09-ba5f-dbe195db1499 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:44:21.789020 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-a6bb0b96-9116-4a09-ba5f-dbe195db1499 None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:44:21.790408 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-a6bb0b96-9116-4a09-ba5f-dbe195db1499 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:44:21.855126 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1170/2358] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 1 16:44:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 16:44:21.864601 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441ce5f2-0b32-4e57-8a59-2f79b89321c4 admin admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:44:21.865999 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-441ce5f2-0b32-4e57-8a59-2f79b89321c4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:44:21.926285 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441ce5f2-0b32-4e57-8a59-2f79b89321c4 admin admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:44:21.929968 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78dc92b8-bade-44d9-a129-ffbd7694ebe0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ed200ce29684c5f80354fa9a8638fe2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0788e6725a84e57a4174284c1845755', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:44:21.930644 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78dc92b8-bade-44d9-a129-ffbd7694ebe0 service neutron] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:44:21.930930 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78dc92b8-bade-44d9-a129-ffbd7694ebe0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:44:21.931207 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78dc92b8-bade-44d9-a129-ffbd7694ebe0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:44:21.942632 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78dc92b8-bade-44d9-a129-ffbd7694ebe0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:44:21.944266 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78dc92b8-bade-44d9-a129-ffbd7694ebe0 service neutron] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:44:21.957381 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1189/2359] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 1 16:44:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 16:44:22.258803 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6bb0b96-9116-4a09-ba5f-dbe195db1499 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:44:22.266103 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-a6bb0b96-9116-4a09-ba5f-dbe195db1499 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:44:22.285452 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6bb0b96-9116-4a09-ba5f-dbe195db1499 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:44:22.291498 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79335ac3-9d9a-42ad-b299-e53629889044 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:44:22.292671 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79335ac3-9d9a-42ad-b299-e53629889044 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:44:22.293214 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79335ac3-9d9a-42ad-b299-e53629889044 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:44:22.293672 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79335ac3-9d9a-42ad-b299-e53629889044 admin admin] PATH_INFO: `/v3/projects/a2ac8b6f02c7469bb7d60dfe07e4ee9c` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:44:22.295017 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79335ac3-9d9a-42ad-b299-e53629889044 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a2ac8b6f02c7469bb7d60dfe07e4ee9c)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:44:22.297076 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79335ac3-9d9a-42ad-b299-e53629889044 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:44:22.345357 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-79335ac3-9d9a-42ad-b299-e53629889044 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a2ac8b6f02c7469bb7d60dfe07e4ee9c', 'request_id': 'req-79335ac3-9d9a-42ad-b299-e53629889044'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:44:22.350792 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-79335ac3-9d9a-42ad-b299-e53629889044 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a2ac8b6f02c7469bb7d60dfe07e4ee9c', 'request_id': 'req-79335ac3-9d9a-42ad-b299-e53629889044'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:44:22.357540 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-79335ac3-9d9a-42ad-b299-e53629889044 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a2ac8b6f02c7469bb7d60dfe07e4ee9c', 'request_id': 'req-79335ac3-9d9a-42ad-b299-e53629889044'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:44:22.373914 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-79335ac3-9d9a-42ad-b299-e53629889044 admin admin] The token cache is being invalidate because project a2ac8b6f02c7469bb7d60dfe07e4ee9c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80311) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 01 16:44:22.374224 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-79335ac3-9d9a-42ad-b299-e53629889044 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:44:22.375120 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-79335ac3-9d9a-42ad-b299-e53629889044 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:44:22.376681 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1171/2360] 10.222.0.57 () {58 vars in 1232 bytes} [Fri Jul 1 16:44:22 2022] DELETE /identity/v3/projects/a2ac8b6f02c7469bb7d60dfe07e4ee9c => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 16:44:31.952476 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=79,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:44:32.368611 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=79,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:44:33.019020 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78dc92b8-bade-44d9-a129-ffbd7694ebe0 service neutron] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:44:33.020543 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-78dc92b8-bade-44d9-a129-ffbd7694ebe0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:44:33.034012 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78dc92b8-bade-44d9-a129-ffbd7694ebe0 service neutron] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:44:33.094787 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de69446-fe72-43b5-aec7-9a46c26ce705 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:44:33.095776 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de69446-fe72-43b5-aec7-9a46c26ce705 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:44:33.096139 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de69446-fe72-43b5-aec7-9a46c26ce705 admin admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:44:33.096493 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de69446-fe72-43b5-aec7-9a46c26ce705 admin admin] PATH_INFO: `/v3/users/854a2f9474a54c229e524d5da7537118` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:44:33.097793 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de69446-fe72-43b5-aec7-9a46c26ce705 admin admin] RBAC: Authorizing `identity:delete_user(user_id=854a2f9474a54c229e524d5da7537118)` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:44:33.099897 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de69446-fe72-43b5-aec7-9a46c26ce705 admin admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:44:33.149485 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-1de69446-fe72-43b5-aec7-9a46c26ce705 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '854a2f9474a54c229e524d5da7537118'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:44:33.162332 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-1de69446-fe72-43b5-aec7-9a46c26ce705 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '854a2f9474a54c229e524d5da7537118'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:44:33.169085 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-1de69446-fe72-43b5-aec7-9a46c26ce705 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '854a2f9474a54c229e524d5da7537118'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:44:33.169800 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-1de69446-fe72-43b5-aec7-9a46c26ce705 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '854a2f9474a54c229e524d5da7537118'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:44:33.170448 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-1de69446-fe72-43b5-aec7-9a46c26ce705 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '854a2f9474a54c229e524d5da7537118'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:44:33.184330 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1190/2361] 10.222.0.57 () {58 vars in 1223 bytes} [Fri Jul 1 16:44:33 2022] DELETE /identity/v3/users/854a2f9474a54c229e524d5da7537118 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 16:44:33.188894 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b771e8a-d2d7-400c-8416-c37a87a3f4bd None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:44:33.189207 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b771e8a-d2d7-400c-8416-c37a87a3f4bd None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:44:33.189537 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b771e8a-d2d7-400c-8416-c37a87a3f4bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:44:33.210109 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-5b771e8a-d2d7-400c-8416-c37a87a3f4bd None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:44:33.211527 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-5b771e8a-d2d7-400c-8416-c37a87a3f4bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:44:33.278446 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1172/2362] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 1 16:44:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 16:44:33.288427 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de69446-fe72-43b5-aec7-9a46c26ce705 admin admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:44:33.289870 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-1de69446-fe72-43b5-aec7-9a46c26ce705 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:44:33.355338 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de69446-fe72-43b5-aec7-9a46c26ce705 admin admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:44:33.358966 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01684fa1-759d-4afc-9350-d45d37e6cdc7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ed200ce29684c5f80354fa9a8638fe2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0788e6725a84e57a4174284c1845755', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:44:33.359601 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01684fa1-759d-4afc-9350-d45d37e6cdc7 service neutron] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:44:33.361781 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01684fa1-759d-4afc-9350-d45d37e6cdc7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:44:33.361781 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01684fa1-759d-4afc-9350-d45d37e6cdc7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:44:33.371077 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01684fa1-759d-4afc-9350-d45d37e6cdc7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:44:33.372537 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01684fa1-759d-4afc-9350-d45d37e6cdc7 service neutron] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:44:33.382130 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1191/2363] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 1 16:44:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 16:44:33.656696 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b771e8a-d2d7-400c-8416-c37a87a3f4bd None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:44:33.658325 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-5b771e8a-d2d7-400c-8416-c37a87a3f4bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:44:33.671331 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b771e8a-d2d7-400c-8416-c37a87a3f4bd None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:44:33.676074 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b4881c-99af-4f2b-a3dc-09efd88662c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:44:33.676973 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b4881c-99af-4f2b-a3dc-09efd88662c8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:44:33.677259 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b4881c-99af-4f2b-a3dc-09efd88662c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:44:33.677534 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b4881c-99af-4f2b-a3dc-09efd88662c8 admin admin] PATH_INFO: `/v3/projects/a5859bbd52284320951b202559ea2e51` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:44:33.678888 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b4881c-99af-4f2b-a3dc-09efd88662c8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a5859bbd52284320951b202559ea2e51)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:44:33.680820 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b4881c-99af-4f2b-a3dc-09efd88662c8 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:44:33.730536 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-c3b4881c-99af-4f2b-a3dc-09efd88662c8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a5859bbd52284320951b202559ea2e51', 'request_id': 'req-c3b4881c-99af-4f2b-a3dc-09efd88662c8'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:44:33.735858 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-c3b4881c-99af-4f2b-a3dc-09efd88662c8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a5859bbd52284320951b202559ea2e51', 'request_id': 'req-c3b4881c-99af-4f2b-a3dc-09efd88662c8'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:44:33.741895 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-c3b4881c-99af-4f2b-a3dc-09efd88662c8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a5859bbd52284320951b202559ea2e51', 'request_id': 'req-c3b4881c-99af-4f2b-a3dc-09efd88662c8'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:44:33.754875 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-c3b4881c-99af-4f2b-a3dc-09efd88662c8 admin admin] The token cache is being invalidate because project a5859bbd52284320951b202559ea2e51 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80311) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 01 16:44:33.755165 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-c3b4881c-99af-4f2b-a3dc-09efd88662c8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:44:33.755789 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-c3b4881c-99af-4f2b-a3dc-09efd88662c8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:44:33.757473 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1173/2364] 10.222.0.57 () {58 vars in 1232 bytes} [Fri Jul 1 16:44:33 2022] DELETE /identity/v3/projects/a5859bbd52284320951b202559ea2e51 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 16:44:33.763023 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232498b3-f2cf-4ac6-9064-2cf3b9c35589 None None] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:44:33.763311 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232498b3-f2cf-4ac6-9064-2cf3b9c35589 None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:44:33.763590 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232498b3-f2cf-4ac6-9064-2cf3b9c35589 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:44:33.781888 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.auth.core [None req-232498b3-f2cf-4ac6-9064-2cf3b9c35589 None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80312) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:44:33.783114 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-232498b3-f2cf-4ac6-9064-2cf3b9c35589 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:44:33.800421 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:44:33.848436 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1192/2365] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 1 16:44:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 16:44:33.857095 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9136c9-e79d-48d1-a297-6bece7b961d6 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:44:33.857387 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9136c9-e79d-48d1-a297-6bece7b961d6 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:44:33.857667 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9136c9-e79d-48d1-a297-6bece7b961d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:44:33.875771 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-da9136c9-e79d-48d1-a297-6bece7b961d6 None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:44:33.876960 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-da9136c9-e79d-48d1-a297-6bece7b961d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:44:33.885400 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1174/2366] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 1 16:44:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 16:44:33.889854 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232498b3-f2cf-4ac6-9064-2cf3b9c35589 None None] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:44:33.899461 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232498b3-f2cf-4ac6-9064-2cf3b9c35589 None None] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:44:33.904784 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ab8144-b863-43d9-a15a-ab9a26d85f14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:44:33.905439 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ab8144-b863-43d9-a15a-ab9a26d85f14 admin admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:44:33.905726 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ab8144-b863-43d9-a15a-ab9a26d85f14 admin admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:44:33.906005 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ab8144-b863-43d9-a15a-ab9a26d85f14 admin admin] PATH_INFO: `/v3/domains` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:44:33.906856 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ab8144-b863-43d9-a15a-ab9a26d85f14 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80312) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:44:33.907201 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ab8144-b863-43d9-a15a-ab9a26d85f14 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:44:33.908821 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ab8144-b863-43d9-a15a-ab9a26d85f14 admin admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:44:33.920926 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1193/2367] 10.222.0.57 () {58 vars in 1152 bytes} [Fri Jul 1 16:44:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:44:33.925235 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9136c9-e79d-48d1-a297-6bece7b961d6 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:44:33.934607 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9136c9-e79d-48d1-a297-6bece7b961d6 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:44:33.939407 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43331d6-7177-4d62-b0dc-e064e2463df0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:44:33.940065 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43331d6-7177-4d62-b0dc-e064e2463df0 admin admin] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:44:33.940345 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43331d6-7177-4d62-b0dc-e064e2463df0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:44:33.940622 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43331d6-7177-4d62-b0dc-e064e2463df0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:44:33.941325 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a43331d6-7177-4d62-b0dc-e064e2463df0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:44:33.943290 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a43331d6-7177-4d62-b0dc-e064e2463df0 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:44:33.970654 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1175/2368] 10.222.0.57 () {60 vars in 1152 bytes} [Fri Jul 1 16:44:33 2022] POST /identity/v3/projects => generated 377 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 01 16:44:33.974821 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ab8144-b863-43d9-a15a-ab9a26d85f14 admin admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:44:33.984861 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ab8144-b863-43d9-a15a-ab9a26d85f14 admin admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:44:34.043534 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2b8c97-4c89-477e-8838-396c0589195a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:44:34.044229 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2b8c97-4c89-477e-8838-396c0589195a admin admin] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:44:34.044485 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2b8c97-4c89-477e-8838-396c0589195a admin admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:44:34.044888 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2b8c97-4c89-477e-8838-396c0589195a admin admin] PATH_INFO: `/v3/users` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:44:34.045589 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea2b8c97-4c89-477e-8838-396c0589195a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:44:34.047617 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea2b8c97-4c89-477e-8838-396c0589195a admin admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:44:34.068018 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1194/2369] 10.222.0.57 () {60 vars in 1143 bytes} [Fri Jul 1 16:44:33 2022] POST /identity/v3/users => generated 289 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 16:44:34.072935 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43331d6-7177-4d62-b0dc-e064e2463df0 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:44:34.083394 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43331d6-7177-4d62-b0dc-e064e2463df0 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:44:34.088514 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f9b02b-492d-4e84-87cf-a3bd010db1bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:44:34.089170 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f9b02b-492d-4e84-87cf-a3bd010db1bd admin admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:44:34.089483 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f9b02b-492d-4e84-87cf-a3bd010db1bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:44:34.089730 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f9b02b-492d-4e84-87cf-a3bd010db1bd admin admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:44:34.090495 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f9b02b-492d-4e84-87cf-a3bd010db1bd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:44:34.092087 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f9b02b-492d-4e84-87cf-a3bd010db1bd admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:44:34.102230 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1176/2370] 10.222.0.57 () {58 vars in 1121 bytes} [Fri Jul 1 16:44:34 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 16:44:34.106399 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2b8c97-4c89-477e-8838-396c0589195a admin admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:44:34.116249 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2b8c97-4c89-477e-8838-396c0589195a admin admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:44:34.121552 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d24a400-c02f-4232-afb2-74a007970f40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:44:34.122330 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d24a400-c02f-4232-afb2-74a007970f40 admin admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:44:34.122663 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d24a400-c02f-4232-afb2-74a007970f40 admin admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:44:34.122949 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d24a400-c02f-4232-afb2-74a007970f40 admin admin] PATH_INFO: `/v3/roles` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:44:34.123699 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d24a400-c02f-4232-afb2-74a007970f40 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:44:34.125444 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d24a400-c02f-4232-afb2-74a007970f40 admin admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:44:34.136510 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1195/2371] 10.222.0.57 () {58 vars in 1121 bytes} [Fri Jul 1 16:44:34 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 16:44:34.140655 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f9b02b-492d-4e84-87cf-a3bd010db1bd admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:44:34.151013 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f9b02b-492d-4e84-87cf-a3bd010db1bd admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:44:34.156139 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d61cfe-4f6e-474d-9fa9-512e1ff4cc27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:44:34.157022 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d61cfe-4f6e-474d-9fa9-512e1ff4cc27 admin admin] REQUEST_METHOD: `PUT` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:44:34.157372 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d61cfe-4f6e-474d-9fa9-512e1ff4cc27 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:44:34.157633 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d61cfe-4f6e-474d-9fa9-512e1ff4cc27 admin admin] PATH_INFO: `/v3/projects/0076c4e65bce4bd58478d09d62607981/users/d6589e2636dd46838a3a0c66d18e0d00/roles/983be40bdab04c478eaa90c6b6a46b14` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:44:34.172828 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d61cfe-4f6e-474d-9fa9-512e1ff4cc27 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0076c4e65bce4bd58478d09d62607981, user_id=d6589e2636dd46838a3a0c66d18e0d00, role_id=983be40bdab04c478eaa90c6b6a46b14)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:44:34.175511 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d61cfe-4f6e-474d-9fa9-512e1ff4cc27 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:44:34.188994 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1177/2372] 10.222.0.57 () {60 vars in 1482 bytes} [Fri Jul 1 16:44:34 2022] PUT /identity/v3/projects/0076c4e65bce4bd58478d09d62607981/users/d6589e2636dd46838a3a0c66d18e0d00/roles/983be40bdab04c478eaa90c6b6a46b14 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 01 16:44:34.194006 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6b5ba0-987f-410d-aa6e-dc8a495c437d None None] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:44:34.194297 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6b5ba0-987f-410d-aa6e-dc8a495c437d None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:44:34.194618 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6b5ba0-987f-410d-aa6e-dc8a495c437d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:44:34.214410 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.auth.core [None req-5e6b5ba0-987f-410d-aa6e-dc8a495c437d None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80312) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:44:34.215648 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-5e6b5ba0-987f-410d-aa6e-dc8a495c437d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:44:34.273175 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1196/2373] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 1 16:44:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 16:44:34.283401 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d61cfe-4f6e-474d-9fa9-512e1ff4cc27 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:44:34.285388 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-c9d61cfe-4f6e-474d-9fa9-512e1ff4cc27 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:44:34.308521 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:44:34.346053 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d61cfe-4f6e-474d-9fa9-512e1ff4cc27 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:44:34.350130 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4084c44f-fd1f-4060-8ab4-f81f23f7eaf0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ed200ce29684c5f80354fa9a8638fe2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0788e6725a84e57a4174284c1845755', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:44:34.350947 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4084c44f-fd1f-4060-8ab4-f81f23f7eaf0 service neutron] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:44:34.351321 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4084c44f-fd1f-4060-8ab4-f81f23f7eaf0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:44:34.351648 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4084c44f-fd1f-4060-8ab4-f81f23f7eaf0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:44:34.363464 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4084c44f-fd1f-4060-8ab4-f81f23f7eaf0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:44:34.365145 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4084c44f-fd1f-4060-8ab4-f81f23f7eaf0 service neutron] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:44:34.376848 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1178/2374] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 1 16:44:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 16:44:38.838147 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1991b2c-ac93-4176-99ef-eefe1d55ee60 None None] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:44:38.839209 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1991b2c-ac93-4176-99ef-eefe1d55ee60 None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:44:38.839414 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1991b2c-ac93-4176-99ef-eefe1d55ee60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:44:38.869494 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.auth.core [None req-d1991b2c-ac93-4176-99ef-eefe1d55ee60 None None] MFA Rules not processed for user `d6589e2636dd46838a3a0c66d18e0d00`. Rule list: `[]` (Enabled: `True`). {{(pid=80312) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:44:38.870756 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-d1991b2c-ac93-4176-99ef-eefe1d55ee60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:44:38.892025 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:44:38.945787 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1197/2375] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 1 16:44:38 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 16:44:38.950852 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4084c44f-fd1f-4060-8ab4-f81f23f7eaf0 service neutron] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:44:38.965487 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4084c44f-fd1f-4060-8ab4-f81f23f7eaf0 service neutron] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:44:38.970439 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07ae57e-1d2e-4764-829d-fd9344acdaf4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:44:38.971150 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07ae57e-1d2e-4764-829d-fd9344acdaf4 admin admin] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:44:38.971435 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07ae57e-1d2e-4764-829d-fd9344acdaf4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:44:38.971721 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07ae57e-1d2e-4764-829d-fd9344acdaf4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:44:38.972521 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b07ae57e-1d2e-4764-829d-fd9344acdaf4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:44:38.974421 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b07ae57e-1d2e-4764-829d-fd9344acdaf4 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:44:39.006956 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1179/2376] 10.222.0.57 () {60 vars in 1152 bytes} [Fri Jul 1 16:44:38 2022] POST /identity/v3/projects => generated 377 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 01 16:44:39.011764 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1991b2c-ac93-4176-99ef-eefe1d55ee60 None None] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:44:39.022941 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1991b2c-ac93-4176-99ef-eefe1d55ee60 None None] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:44:39.083349 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b01f24-07c2-474c-8be9-19f1434a5fb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:44:39.084048 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b01f24-07c2-474c-8be9-19f1434a5fb6 admin admin] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:44:39.084589 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b01f24-07c2-474c-8be9-19f1434a5fb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:44:39.084862 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b01f24-07c2-474c-8be9-19f1434a5fb6 admin admin] PATH_INFO: `/v3/users` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:44:39.085594 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b01f24-07c2-474c-8be9-19f1434a5fb6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:44:39.087627 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b01f24-07c2-474c-8be9-19f1434a5fb6 admin admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:44:39.112486 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1198/2377] 10.222.0.57 () {60 vars in 1143 bytes} [Fri Jul 1 16:44:39 2022] POST /identity/v3/users => generated 295 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 16:44:39.116168 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07ae57e-1d2e-4764-829d-fd9344acdaf4 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:44:39.133959 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07ae57e-1d2e-4764-829d-fd9344acdaf4 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:44:39.142034 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2daf44-9259-41ac-b7f0-ebd495139a6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:44:39.142783 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2daf44-9259-41ac-b7f0-ebd495139a6a admin admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:44:39.143072 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2daf44-9259-41ac-b7f0-ebd495139a6a admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:44:39.143353 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2daf44-9259-41ac-b7f0-ebd495139a6a admin admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:44:39.144099 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c2daf44-9259-41ac-b7f0-ebd495139a6a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:44:39.145720 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c2daf44-9259-41ac-b7f0-ebd495139a6a admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:44:39.157330 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1180/2378] 10.222.0.57 () {58 vars in 1121 bytes} [Fri Jul 1 16:44:39 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 01 16:44:39.161783 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b01f24-07c2-474c-8be9-19f1434a5fb6 admin admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:44:39.173726 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b01f24-07c2-474c-8be9-19f1434a5fb6 admin admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:44:39.178493 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f78f6d-beaf-48d1-b316-011f9a522985 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:44:39.179514 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f78f6d-beaf-48d1-b316-011f9a522985 admin admin] REQUEST_METHOD: `PUT` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:44:39.179797 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f78f6d-beaf-48d1-b316-011f9a522985 admin admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:44:39.180072 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f78f6d-beaf-48d1-b316-011f9a522985 admin admin] PATH_INFO: `/v3/projects/bb7f4747ad734029884a38be8a3adfbf/users/296adde7ece444e391cedf9710323c44/roles/84ee739146934c1ab32c501ff8093644` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:44:39.196190 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f78f6d-beaf-48d1-b316-011f9a522985 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bb7f4747ad734029884a38be8a3adfbf, user_id=296adde7ece444e391cedf9710323c44, role_id=84ee739146934c1ab32c501ff8093644)` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:44:39.199847 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f78f6d-beaf-48d1-b316-011f9a522985 admin admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:44:39.214917 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1199/2379] 10.222.0.57 () {60 vars in 1482 bytes} [Fri Jul 1 16:44:39 2022] PUT /identity/v3/projects/bb7f4747ad734029884a38be8a3adfbf/users/296adde7ece444e391cedf9710323c44/roles/84ee739146934c1ab32c501ff8093644 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 16:44:39.219189 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2daf44-9259-41ac-b7f0-ebd495139a6a admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:44:39.229497 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2daf44-9259-41ac-b7f0-ebd495139a6a admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:44:39.299662 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2bc8c69-8044-40cc-b971-c159b912d11c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:44:39.300342 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bc8c69-8044-40cc-b971-c159b912d11c admin admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:44:39.300654 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bc8c69-8044-40cc-b971-c159b912d11c admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:44:39.300938 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bc8c69-8044-40cc-b971-c159b912d11c admin admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:44:39.301684 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2bc8c69-8044-40cc-b971-c159b912d11c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:44:39.303342 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2bc8c69-8044-40cc-b971-c159b912d11c admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:44:39.315305 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1181/2380] 10.222.0.57 () {58 vars in 1121 bytes} [Fri Jul 1 16:44:39 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 16:44:39.319559 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f78f6d-beaf-48d1-b316-011f9a522985 admin admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:44:39.330988 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f78f6d-beaf-48d1-b316-011f9a522985 admin admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:44:39.335941 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a584076b-d5c7-4f02-855d-d6e4643c4ff4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:44:39.336832 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a584076b-d5c7-4f02-855d-d6e4643c4ff4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:44:39.337116 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a584076b-d5c7-4f02-855d-d6e4643c4ff4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:44:39.337396 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a584076b-d5c7-4f02-855d-d6e4643c4ff4 admin admin] PATH_INFO: `/v3/projects/bb7f4747ad734029884a38be8a3adfbf/users/296adde7ece444e391cedf9710323c44/roles/84ee739146934c1ab32c501ff8093644` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:44:39.339567 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a584076b-d5c7-4f02-855d-d6e4643c4ff4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bb7f4747ad734029884a38be8a3adfbf, user_id=296adde7ece444e391cedf9710323c44, role_id=84ee739146934c1ab32c501ff8093644)` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:44:39.342250 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a584076b-d5c7-4f02-855d-d6e4643c4ff4 admin admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:44:39.354434 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1200/2381] 10.222.0.57 () {60 vars in 1482 bytes} [Fri Jul 1 16:44:39 2022] PUT /identity/v3/projects/bb7f4747ad734029884a38be8a3adfbf/users/296adde7ece444e391cedf9710323c44/roles/84ee739146934c1ab32c501ff8093644 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 16:44:43.977692 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456a153d-9f08-4aa9-988c-36c4a5e8f50d None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:44:43.978244 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456a153d-9f08-4aa9-988c-36c4a5e8f50d None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:44:43.978339 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456a153d-9f08-4aa9-988c-36c4a5e8f50d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:44:44.006878 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-456a153d-9f08-4aa9-988c-36c4a5e8f50d None None] MFA Rules not processed for user `296adde7ece444e391cedf9710323c44`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:44:44.008119 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-456a153d-9f08-4aa9-988c-36c4a5e8f50d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:44:44.079005 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1182/2382] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 1 16:44:43 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 16:44:44.093286 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a584076b-d5c7-4f02-855d-d6e4643c4ff4 admin admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:44:44.095069 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-a584076b-d5c7-4f02-855d-d6e4643c4ff4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:44:44.183821 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:44:44.212700 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a584076b-d5c7-4f02-855d-d6e4643c4ff4 admin admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:44:44.218918 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec95ee6-6cbd-4bc1-a3de-4075f5adc297 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a50f745b31941b7a7265c0fc852f48e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0788e6725a84e57a4174284c1845755', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:44:44.219569 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec95ee6-6cbd-4bc1-a3de-4075f5adc297 service nova] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:44:44.219855 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec95ee6-6cbd-4bc1-a3de-4075f5adc297 service nova] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:44:44.220133 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec95ee6-6cbd-4bc1-a3de-4075f5adc297 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:44:44.280518 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec95ee6-6cbd-4bc1-a3de-4075f5adc297 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:44:44.282072 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec95ee6-6cbd-4bc1-a3de-4075f5adc297 service nova] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:44:44.291937 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1201/2383] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 1 16:44:44 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 199 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 01 16:44:47.157422 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a52c3da-3a4c-43be-bb85-b2c983785cb7 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:44:47.157862 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a52c3da-3a4c-43be-bb85-b2c983785cb7 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:44:47.158317 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a52c3da-3a4c-43be-bb85-b2c983785cb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:44:47.184856 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-2a52c3da-3a4c-43be-bb85-b2c983785cb7 None None] MFA Rules not processed for user `71c383c04f9543d8ab97a6c0db268ca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:44:47.187067 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-2a52c3da-3a4c-43be-bb85-b2c983785cb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:44:47.195488 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:44:47.265344 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1183/2384] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 1 16:44:47 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 16:44:54.287071 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=33 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:44:56.948927 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8764b27e-969d-4f62-b571-14dca9f3dcce None None] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:44:56.949502 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8764b27e-969d-4f62-b571-14dca9f3dcce None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:44:56.949502 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8764b27e-969d-4f62-b571-14dca9f3dcce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:44:56.966972 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.auth.core [None req-8764b27e-969d-4f62-b571-14dca9f3dcce None None] MFA Rules not processed for user `e471b31763074167ba81c2c25f39b857`. Rule list: `[]` (Enabled: `True`). {{(pid=80312) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:44:56.968098 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-8764b27e-969d-4f62-b571-14dca9f3dcce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:44:57.029619 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1202/2385] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 1 16:44:56 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 16:44:57.256392 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=20 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:45:04.177991 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99d4fad-b2c7-40a4-9646-420e9bd539dc None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:45:04.178635 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99d4fad-b2c7-40a4-9646-420e9bd539dc None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:45:04.178635 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99d4fad-b2c7-40a4-9646-420e9bd539dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:45:04.196125 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-d99d4fad-b2c7-40a4-9646-420e9bd539dc None None] MFA Rules not processed for user `d6589e2636dd46838a3a0c66d18e0d00`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:45:04.197321 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-d99d4fad-b2c7-40a4-9646-420e9bd539dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:45:04.204202 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1184/2386] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 1 16:45:04 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 27 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 16:45:07.022068 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=25 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:45:14.191017 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=3 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:45:17.209417 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8764b27e-969d-4f62-b571-14dca9f3dcce None None] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:45:17.212137 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-8764b27e-969d-4f62-b571-14dca9f3dcce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:45:17.269577 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8764b27e-969d-4f62-b571-14dca9f3dcce None None] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:45:17.273994 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acfabb3d-cca4-400c-8b9c-0187ca0a9fc0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ed200ce29684c5f80354fa9a8638fe2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0788e6725a84e57a4174284c1845755', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:45:17.274701 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfabb3d-cca4-400c-8b9c-0187ca0a9fc0 service neutron] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:45:17.274991 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfabb3d-cca4-400c-8b9c-0187ca0a9fc0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:45:17.275276 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfabb3d-cca4-400c-8b9c-0187ca0a9fc0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:45:17.277224 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-acfabb3d-cca4-400c-8b9c-0187ca0a9fc0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:45:17.292630 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acfabb3d-cca4-400c-8b9c-0187ca0a9fc0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:45:17.294103 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acfabb3d-cca4-400c-8b9c-0187ca0a9fc0 service neutron] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:45:17.306801 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1203/2387] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 1 16:45:17 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 16:45:20.308269 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99d4fad-b2c7-40a4-9646-420e9bd539dc None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:45:20.309668 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-d99d4fad-b2c7-40a4-9646-420e9bd539dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:45:20.373572 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99d4fad-b2c7-40a4-9646-420e9bd539dc None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:45:20.377256 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc116f4-e8bf-4399-85bb-aa95f619447d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbddf971a2354bbdbb9da1d63f3abf88', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0788e6725a84e57a4174284c1845755', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:45:20.377911 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc116f4-e8bf-4399-85bb-aa95f619447d service cinder] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:45:20.378197 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc116f4-e8bf-4399-85bb-aa95f619447d service cinder] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:45:20.378510 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc116f4-e8bf-4399-85bb-aa95f619447d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:45:20.380314 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-fcc116f4-e8bf-4399-85bb-aa95f619447d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:45:20.394767 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc116f4-e8bf-4399-85bb-aa95f619447d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:45:20.396195 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc116f4-e8bf-4399-85bb-aa95f619447d service cinder] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:45:20.408072 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1185/2388] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 1 16:45:20 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 16:45:22.606166 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acfabb3d-cca4-400c-8b9c-0187ca0a9fc0 service neutron] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:45:22.616152 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acfabb3d-cca4-400c-8b9c-0187ca0a9fc0 service neutron] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:45:22.620574 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d57e960-52f9-4b3f-abc0-c30bdc08ce8a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbddf971a2354bbdbb9da1d63f3abf88', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0788e6725a84e57a4174284c1845755', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:45:22.621307 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d57e960-52f9-4b3f-abc0-c30bdc08ce8a service cinder] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:45:22.621596 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d57e960-52f9-4b3f-abc0-c30bdc08ce8a service cinder] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:45:22.621870 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d57e960-52f9-4b3f-abc0-c30bdc08ce8a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:45:22.624092 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-6d57e960-52f9-4b3f-abc0-c30bdc08ce8a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:45:22.685239 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d57e960-52f9-4b3f-abc0-c30bdc08ce8a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:45:22.686722 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d57e960-52f9-4b3f-abc0-c30bdc08ce8a service cinder] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:45:22.696311 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1204/2389] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 1 16:45:22 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 16:45:24.933749 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc116f4-e8bf-4399-85bb-aa95f619447d service cinder] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:45:24.944243 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc116f4-e8bf-4399-85bb-aa95f619447d service cinder] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:45:24.949079 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9055910-901c-48b3-83ed-19e799668620 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ed200ce29684c5f80354fa9a8638fe2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0788e6725a84e57a4174284c1845755', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:45:24.950226 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9055910-901c-48b3-83ed-19e799668620 service neutron] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:45:24.950890 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9055910-901c-48b3-83ed-19e799668620 service neutron] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:45:24.951357 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9055910-901c-48b3-83ed-19e799668620 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:45:24.965640 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9055910-901c-48b3-83ed-19e799668620 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:45:24.967300 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9055910-901c-48b3-83ed-19e799668620 service neutron] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:45:24.978949 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1186/2390] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 1 16:45:24 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 16:45:24.991727 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e64edb-11a2-4ab0-9ecb-41de031a7193 None None] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:45:24.992222 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e64edb-11a2-4ab0-9ecb-41de031a7193 None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:45:24.992330 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e64edb-11a2-4ab0-9ecb-41de031a7193 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:45:25.011600 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.auth.core [None req-28e64edb-11a2-4ab0-9ecb-41de031a7193 None None] MFA Rules not processed for user `c4de1b1eeaed4fc5ae7200b2c7a90957`. Rule list: `[]` (Enabled: `True`). {{(pid=80312) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:45:25.012802 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-28e64edb-11a2-4ab0-9ecb-41de031a7193 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:45:25.080437 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1205/2391] 10.222.0.57 () {58 vars in 967 bytes} [Fri Jul 1 16:45:24 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 16:45:25.155417 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9055910-901c-48b3-83ed-19e799668620 service neutron] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:45:25.156863 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-c9055910-901c-48b3-83ed-19e799668620 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:45:25.230444 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9055910-901c-48b3-83ed-19e799668620 service neutron] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:45:25.235700 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4095e8df-2daa-4f95-a2b3-392b90bcd453 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '859fe410c2d2450fb803c6244839f27c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0788e6725a84e57a4174284c1845755', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:45:25.236366 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4095e8df-2daa-4f95-a2b3-392b90bcd453 service swift] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:45:25.236658 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4095e8df-2daa-4f95-a2b3-392b90bcd453 service swift] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:45:25.236957 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4095e8df-2daa-4f95-a2b3-392b90bcd453 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:45:25.246117 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4095e8df-2daa-4f95-a2b3-392b90bcd453 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:45:25.247773 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4095e8df-2daa-4f95-a2b3-392b90bcd453 service swift] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:45:25.256048 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1187/2392] 10.222.0.57 () {60 vars in 1384 bytes} [Fri Jul 1 16:45:25 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 01 16:45:33.370387 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e64edb-11a2-4ab0-9ecb-41de031a7193 None None] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:45:33.371848 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-28e64edb-11a2-4ab0-9ecb-41de031a7193 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:45:33.431068 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4095e8df-2daa-4f95-a2b3-392b90bcd453 service swift] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:45:33.432494 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-4095e8df-2daa-4f95-a2b3-392b90bcd453 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:45:33.434791 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=100 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:45:33.461603 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e64edb-11a2-4ab0-9ecb-41de031a7193 None None] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:45:33.466907 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f755c1b-ea68-471a-b794-3fc37db4f005 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc38af18dec4e9b8143df23af6aad43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0788e6725a84e57a4174284c1845755', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:45:33.467650 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f755c1b-ea68-471a-b794-3fc37db4f005 service placement] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:45:33.467941 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f755c1b-ea68-471a-b794-3fc37db4f005 service placement] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:45:33.468220 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f755c1b-ea68-471a-b794-3fc37db4f005 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:45:33.470007 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-1f755c1b-ea68-471a-b794-3fc37db4f005 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:45:33.482766 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f755c1b-ea68-471a-b794-3fc37db4f005 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:45:33.484426 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f755c1b-ea68-471a-b794-3fc37db4f005 service placement] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:45:33.496869 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1206/2393] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 1 16:45:33 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 16:45:33.502690 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4095e8df-2daa-4f95-a2b3-392b90bcd453 service swift] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:45:33.507962 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581cc544-0397-4e3c-a619-fc6b7366b227 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:45:33.508826 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581cc544-0397-4e3c-a619-fc6b7366b227 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:45:33.509105 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581cc544-0397-4e3c-a619-fc6b7366b227 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:45:33.509376 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581cc544-0397-4e3c-a619-fc6b7366b227 admin admin] PATH_INFO: `/v3/users/c1ab0b52325c4ab387db034eb7dfcadb` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:45:33.510608 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-581cc544-0397-4e3c-a619-fc6b7366b227 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c1ab0b52325c4ab387db034eb7dfcadb)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:45:33.513060 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-581cc544-0397-4e3c-a619-fc6b7366b227 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:45:33.537948 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:45:33.564333 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-581cc544-0397-4e3c-a619-fc6b7366b227 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c1ab0b52325c4ab387db034eb7dfcadb'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:45:33.564978 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-581cc544-0397-4e3c-a619-fc6b7366b227 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c1ab0b52325c4ab387db034eb7dfcadb'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:45:33.565609 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-581cc544-0397-4e3c-a619-fc6b7366b227 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c1ab0b52325c4ab387db034eb7dfcadb'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:45:33.573884 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-581cc544-0397-4e3c-a619-fc6b7366b227 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c1ab0b52325c4ab387db034eb7dfcadb'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:45:33.586310 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-581cc544-0397-4e3c-a619-fc6b7366b227 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c1ab0b52325c4ab387db034eb7dfcadb'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:45:33.599936 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1188/2394] 10.222.0.57 () {58 vars in 1223 bytes} [Fri Jul 1 16:45:33 2022] DELETE /identity/v3/users/c1ab0b52325c4ab387db034eb7dfcadb => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 01 16:45:33.793391 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f755c1b-ea68-471a-b794-3fc37db4f005 service placement] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:45:33.812074 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f755c1b-ea68-471a-b794-3fc37db4f005 service placement] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:45:33.899206 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b087f27f-d333-4123-b4ee-e19b811ecf6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:45:33.900096 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b087f27f-d333-4123-b4ee-e19b811ecf6b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:45:33.900384 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b087f27f-d333-4123-b4ee-e19b811ecf6b admin admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:45:33.900776 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b087f27f-d333-4123-b4ee-e19b811ecf6b admin admin] PATH_INFO: `/v3/projects/ef0cc672766c47759a088a398d95147f` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:45:33.902036 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b087f27f-d333-4123-b4ee-e19b811ecf6b admin admin] RBAC: Authorizing `identity:delete_project(project_id=ef0cc672766c47759a088a398d95147f)` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:45:33.904006 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b087f27f-d333-4123-b4ee-e19b811ecf6b admin admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:45:33.954686 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-b087f27f-d333-4123-b4ee-e19b811ecf6b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ef0cc672766c47759a088a398d95147f', 'request_id': 'req-b087f27f-d333-4123-b4ee-e19b811ecf6b'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:45:33.963848 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-b087f27f-d333-4123-b4ee-e19b811ecf6b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ef0cc672766c47759a088a398d95147f', 'request_id': 'req-b087f27f-d333-4123-b4ee-e19b811ecf6b'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:45:33.970426 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-b087f27f-d333-4123-b4ee-e19b811ecf6b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ef0cc672766c47759a088a398d95147f', 'request_id': 'req-b087f27f-d333-4123-b4ee-e19b811ecf6b'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:45:33.982408 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-b087f27f-d333-4123-b4ee-e19b811ecf6b admin admin] The token cache is being invalidate because project ef0cc672766c47759a088a398d95147f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80312) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 01 16:45:33.982738 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-b087f27f-d333-4123-b4ee-e19b811ecf6b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:45:33.983408 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-b087f27f-d333-4123-b4ee-e19b811ecf6b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:45:33.984981 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1207/2395] 10.222.0.57 () {58 vars in 1232 bytes} [Fri Jul 1 16:45:33 2022] DELETE /identity/v3/projects/ef0cc672766c47759a088a398d95147f => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 16:45:33.989004 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581cc544-0397-4e3c-a619-fc6b7366b227 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:45:33.990481 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-581cc544-0397-4e3c-a619-fc6b7366b227 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:45:34.005312 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581cc544-0397-4e3c-a619-fc6b7366b227 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:45:34.070456 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d51dd6-f3cb-4ae5-b070-2bd46f879cf0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:45:34.071673 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d51dd6-f3cb-4ae5-b070-2bd46f879cf0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:45:34.071959 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d51dd6-f3cb-4ae5-b070-2bd46f879cf0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:45:34.072235 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d51dd6-f3cb-4ae5-b070-2bd46f879cf0 admin admin] PATH_INFO: `/v3/users/4a269ade40e44ad986f32ce794371e8b` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:45:34.073490 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d51dd6-f3cb-4ae5-b070-2bd46f879cf0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4a269ade40e44ad986f32ce794371e8b)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:45:34.075486 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d51dd6-f3cb-4ae5-b070-2bd46f879cf0 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:45:34.128868 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-44d51dd6-f3cb-4ae5-b070-2bd46f879cf0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4a269ade40e44ad986f32ce794371e8b'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:45:34.129789 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-44d51dd6-f3cb-4ae5-b070-2bd46f879cf0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4a269ade40e44ad986f32ce794371e8b'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:45:34.130425 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-44d51dd6-f3cb-4ae5-b070-2bd46f879cf0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4a269ade40e44ad986f32ce794371e8b'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:45:34.140395 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-44d51dd6-f3cb-4ae5-b070-2bd46f879cf0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4a269ade40e44ad986f32ce794371e8b'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:45:34.152763 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-44d51dd6-f3cb-4ae5-b070-2bd46f879cf0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4a269ade40e44ad986f32ce794371e8b'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:45:34.165365 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1189/2396] 10.222.0.57 () {58 vars in 1223 bytes} [Fri Jul 1 16:45:33 2022] DELETE /identity/v3/users/4a269ade40e44ad986f32ce794371e8b => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 16:45:34.347013 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b087f27f-d333-4123-b4ee-e19b811ecf6b admin admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:45:34.348439 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-b087f27f-d333-4123-b4ee-e19b811ecf6b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:45:34.364014 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b087f27f-d333-4123-b4ee-e19b811ecf6b admin admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:45:34.423592 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fad6ed8-7d97-45db-9f11-897fdaf53ca2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:45:34.424375 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fad6ed8-7d97-45db-9f11-897fdaf53ca2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:45:34.424663 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fad6ed8-7d97-45db-9f11-897fdaf53ca2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:45:34.424944 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fad6ed8-7d97-45db-9f11-897fdaf53ca2 admin admin] PATH_INFO: `/v3/projects/f8d3731c3718400b9f8143151dfaac9f` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:45:34.426160 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fad6ed8-7d97-45db-9f11-897fdaf53ca2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f8d3731c3718400b9f8143151dfaac9f)` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:45:34.428120 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fad6ed8-7d97-45db-9f11-897fdaf53ca2 admin admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:45:34.436689 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:45:34.476965 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-7fad6ed8-7d97-45db-9f11-897fdaf53ca2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f8d3731c3718400b9f8143151dfaac9f', 'request_id': 'req-7fad6ed8-7d97-45db-9f11-897fdaf53ca2'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:45:34.485252 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-7fad6ed8-7d97-45db-9f11-897fdaf53ca2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f8d3731c3718400b9f8143151dfaac9f', 'request_id': 'req-7fad6ed8-7d97-45db-9f11-897fdaf53ca2'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:45:34.491164 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-7fad6ed8-7d97-45db-9f11-897fdaf53ca2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f8d3731c3718400b9f8143151dfaac9f', 'request_id': 'req-7fad6ed8-7d97-45db-9f11-897fdaf53ca2'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:45:34.502972 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-7fad6ed8-7d97-45db-9f11-897fdaf53ca2 admin admin] The token cache is being invalidate because project f8d3731c3718400b9f8143151dfaac9f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80312) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 01 16:45:34.503259 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-7fad6ed8-7d97-45db-9f11-897fdaf53ca2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:45:34.503890 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-7fad6ed8-7d97-45db-9f11-897fdaf53ca2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:45:34.505457 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1208/2397] 10.222.0.57 () {58 vars in 1232 bytes} [Fri Jul 1 16:45:34 2022] DELETE /identity/v3/projects/f8d3731c3718400b9f8143151dfaac9f => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 16:45:39.705960 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bd7ce9-b6ab-48f2-a204-e1d640b7bed7 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:45:39.706550 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bd7ce9-b6ab-48f2-a204-e1d640b7bed7 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:45:39.707063 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bd7ce9-b6ab-48f2-a204-e1d640b7bed7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:45:39.730153 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-83bd7ce9-b6ab-48f2-a204-e1d640b7bed7 None None] MFA Rules not processed for user `7c4da21fbfa3472cac6e766a2f4799d8`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:45:39.731449 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-83bd7ce9-b6ab-48f2-a204-e1d640b7bed7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:45:39.747221 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:DELETE=17,keystone:SELECT=81,keystone:INSERT=2 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:45:39.796347 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1190/2398] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 1 16:45:39 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 16:45:40.514479 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fad6ed8-7d97-45db-9f11-897fdaf53ca2 admin admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:45:40.515985 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-7fad6ed8-7d97-45db-9f11-897fdaf53ca2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:45:40.606210 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fad6ed8-7d97-45db-9f11-897fdaf53ca2 admin admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:45:40.611444 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6739d43-4b96-423b-a9b6-cbd3b6c6dc03 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc38af18dec4e9b8143df23af6aad43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0788e6725a84e57a4174284c1845755', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:45:40.612117 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6739d43-4b96-423b-a9b6-cbd3b6c6dc03 service placement] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:45:40.612407 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6739d43-4b96-423b-a9b6-cbd3b6c6dc03 service placement] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:45:40.612681 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6739d43-4b96-423b-a9b6-cbd3b6c6dc03 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:45:40.614562 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-b6739d43-4b96-423b-a9b6-cbd3b6c6dc03 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:45:40.629275 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6739d43-4b96-423b-a9b6-cbd3b6c6dc03 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:45:40.630950 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6739d43-4b96-423b-a9b6-cbd3b6c6dc03 service placement] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:45:40.642788 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1209/2399] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 1 16:45:40 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 16:45:42.411190 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83bd7ce9-b6ab-48f2-a204-e1d640b7bed7 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:45:42.412568 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-83bd7ce9-b6ab-48f2-a204-e1d640b7bed7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:45:42.472655 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83bd7ce9-b6ab-48f2-a204-e1d640b7bed7 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:45:42.477407 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd21f05-17b4-479b-845c-432b32752964 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ed200ce29684c5f80354fa9a8638fe2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0788e6725a84e57a4174284c1845755', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:45:42.478059 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd21f05-17b4-479b-845c-432b32752964 service neutron] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:45:42.478365 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd21f05-17b4-479b-845c-432b32752964 service neutron] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:45:42.478646 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd21f05-17b4-479b-845c-432b32752964 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:45:42.480403 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-cfd21f05-17b4-479b-845c-432b32752964 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:45:42.541725 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd21f05-17b4-479b-845c-432b32752964 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:45:42.543210 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd21f05-17b4-479b-845c-432b32752964 service neutron] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:45:42.552363 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1191/2400] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 1 16:45:42 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 16:45:46.166736 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6739d43-4b96-423b-a9b6-cbd3b6c6dc03 service placement] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:45:46.168167 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-b6739d43-4b96-423b-a9b6-cbd3b6c6dc03 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:45:46.179964 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6739d43-4b96-423b-a9b6-cbd3b6c6dc03 service placement] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:45:46.183665 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7950e514-0543-498b-99b6-98c26eded65e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ed200ce29684c5f80354fa9a8638fe2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0788e6725a84e57a4174284c1845755', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:45:46.184303 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7950e514-0543-498b-99b6-98c26eded65e service neutron] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:45:46.184587 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7950e514-0543-498b-99b6-98c26eded65e service neutron] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:45:46.184892 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7950e514-0543-498b-99b6-98c26eded65e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:45:46.186667 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-7950e514-0543-498b-99b6-98c26eded65e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:45:46.196276 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7950e514-0543-498b-99b6-98c26eded65e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:45:46.197692 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7950e514-0543-498b-99b6-98c26eded65e service neutron] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:45:46.205505 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1210/2401] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 1 16:45:46 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 16:45:51.160029 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd21f05-17b4-479b-845c-432b32752964 service neutron] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:45:51.161754 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-cfd21f05-17b4-479b-845c-432b32752964 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:45:51.279455 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd21f05-17b4-479b-845c-432b32752964 service neutron] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:45:51.285419 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50779f88-cad4-485b-82af-47793438a71c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a50f745b31941b7a7265c0fc852f48e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0788e6725a84e57a4174284c1845755', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:45:51.286078 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50779f88-cad4-485b-82af-47793438a71c service nova] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:45:51.286389 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50779f88-cad4-485b-82af-47793438a71c service nova] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:45:51.286676 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50779f88-cad4-485b-82af-47793438a71c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:45:51.289921 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-50779f88-cad4-485b-82af-47793438a71c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:45:51.303596 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50779f88-cad4-485b-82af-47793438a71c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:45:51.305456 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50779f88-cad4-485b-82af-47793438a71c service nova] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:45:51.316901 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1192/2402] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 1 16:45:51 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 16:45:55.856573 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7950e514-0543-498b-99b6-98c26eded65e service neutron] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:45:55.857978 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-7950e514-0543-498b-99b6-98c26eded65e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:45:55.874297 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7950e514-0543-498b-99b6-98c26eded65e service neutron] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:45:55.879756 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d050c2-9871-4c6c-95e5-bc412405bd87 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a50f745b31941b7a7265c0fc852f48e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0788e6725a84e57a4174284c1845755', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:45:55.880389 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d050c2-9871-4c6c-95e5-bc412405bd87 service nova] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:45:55.880670 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d050c2-9871-4c6c-95e5-bc412405bd87 service nova] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:45:55.881882 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d050c2-9871-4c6c-95e5-bc412405bd87 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:45:55.883653 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-77d050c2-9871-4c6c-95e5-bc412405bd87 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:45:55.942508 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d050c2-9871-4c6c-95e5-bc412405bd87 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:45:55.944758 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d050c2-9871-4c6c-95e5-bc412405bd87 service nova] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:45:55.953766 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1211/2403] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 1 16:45:55 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 16:45:57.877115 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50779f88-cad4-485b-82af-47793438a71c service nova] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:45:57.884478 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:45:57.891131 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50779f88-cad4-485b-82af-47793438a71c service nova] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:45:57.897516 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eae0eaf-1ca4-483c-8c0b-f5dc21d13828 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a50f745b31941b7a7265c0fc852f48e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0788e6725a84e57a4174284c1845755', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:45:57.898174 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eae0eaf-1ca4-483c-8c0b-f5dc21d13828 service nova] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:45:57.898539 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eae0eaf-1ca4-483c-8c0b-f5dc21d13828 service nova] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:45:57.898816 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eae0eaf-1ca4-483c-8c0b-f5dc21d13828 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:45:57.900813 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-7eae0eaf-1ca4-483c-8c0b-f5dc21d13828 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:45:57.915321 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eae0eaf-1ca4-483c-8c0b-f5dc21d13828 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:45:57.916905 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eae0eaf-1ca4-483c-8c0b-f5dc21d13828 service nova] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:45:57.928475 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1193/2404] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 1 16:45:57 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 16:45:59.722567 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d050c2-9871-4c6c-95e5-bc412405bd87 service nova] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:45:59.732943 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d050c2-9871-4c6c-95e5-bc412405bd87 service nova] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:45:59.736661 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1785d59-e545-400b-afb8-a36fb2f83a58 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ed200ce29684c5f80354fa9a8638fe2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0788e6725a84e57a4174284c1845755', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:45:59.737363 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1785d59-e545-400b-afb8-a36fb2f83a58 service neutron] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:45:59.737653 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1785d59-e545-400b-afb8-a36fb2f83a58 service neutron] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:45:59.737930 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1785d59-e545-400b-afb8-a36fb2f83a58 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:45:59.739750 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-f1785d59-e545-400b-afb8-a36fb2f83a58 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:45:59.750138 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:45:59.821910 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1785d59-e545-400b-afb8-a36fb2f83a58 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:45:59.824230 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1785d59-e545-400b-afb8-a36fb2f83a58 service neutron] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:45:59.838337 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1212/2405] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 1 16:45:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 16:46:02.948614 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eae0eaf-1ca4-483c-8c0b-f5dc21d13828 service nova] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:46:02.950041 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-7eae0eaf-1ca4-483c-8c0b-f5dc21d13828 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:46:02.964549 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eae0eaf-1ca4-483c-8c0b-f5dc21d13828 service nova] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:46:02.970565 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d70737-5947-4105-9ffd-c61c1b0bad7f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc38af18dec4e9b8143df23af6aad43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0788e6725a84e57a4174284c1845755', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:46:02.971221 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d70737-5947-4105-9ffd-c61c1b0bad7f service placement] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:46:02.971508 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d70737-5947-4105-9ffd-c61c1b0bad7f service placement] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:46:02.971786 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d70737-5947-4105-9ffd-c61c1b0bad7f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:46:02.973554 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-e9d70737-5947-4105-9ffd-c61c1b0bad7f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:46:02.986967 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d70737-5947-4105-9ffd-c61c1b0bad7f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:46:02.988809 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d70737-5947-4105-9ffd-c61c1b0bad7f service placement] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:46:03.003174 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1194/2406] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 1 16:46:02 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 16:46:03.012365 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1785d59-e545-400b-afb8-a36fb2f83a58 service neutron] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:46:03.013957 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-f1785d59-e545-400b-afb8-a36fb2f83a58 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:46:03.029165 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1785d59-e545-400b-afb8-a36fb2f83a58 service neutron] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:46:03.035414 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c61e5d8-3758-4655-9d16-480d52d31089 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:46:03.036303 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c61e5d8-3758-4655-9d16-480d52d31089 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:46:03.036590 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c61e5d8-3758-4655-9d16-480d52d31089 admin admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:46:03.037476 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c61e5d8-3758-4655-9d16-480d52d31089 admin admin] PATH_INFO: `/v3/users/71c383c04f9543d8ab97a6c0db268ca5` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:46:03.039645 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c61e5d8-3758-4655-9d16-480d52d31089 admin admin] RBAC: Authorizing `identity:delete_user(user_id=71c383c04f9543d8ab97a6c0db268ca5)` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:46:03.041843 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c61e5d8-3758-4655-9d16-480d52d31089 admin admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:46:03.100993 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-0c61e5d8-3758-4655-9d16-480d52d31089 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '71c383c04f9543d8ab97a6c0db268ca5'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:46:03.113741 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-0c61e5d8-3758-4655-9d16-480d52d31089 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '71c383c04f9543d8ab97a6c0db268ca5'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:46:03.121683 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-0c61e5d8-3758-4655-9d16-480d52d31089 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '71c383c04f9543d8ab97a6c0db268ca5'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:46:03.122462 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-0c61e5d8-3758-4655-9d16-480d52d31089 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '71c383c04f9543d8ab97a6c0db268ca5'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:46:03.123167 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-0c61e5d8-3758-4655-9d16-480d52d31089 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '71c383c04f9543d8ab97a6c0db268ca5'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:46:03.140959 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1213/2407] 10.222.0.57 () {58 vars in 1223 bytes} [Fri Jul 1 16:46:03 2022] DELETE /identity/v3/users/71c383c04f9543d8ab97a6c0db268ca5 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 16:46:03.313657 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d70737-5947-4105-9ffd-c61c1b0bad7f service placement] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:46:03.326795 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d70737-5947-4105-9ffd-c61c1b0bad7f service placement] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:46:03.386642 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f5f7ec-d18b-4c42-8bcb-788722f1526e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:46:03.387509 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f5f7ec-d18b-4c42-8bcb-788722f1526e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:46:03.387796 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f5f7ec-d18b-4c42-8bcb-788722f1526e admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:46:03.388079 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f5f7ec-d18b-4c42-8bcb-788722f1526e admin admin] PATH_INFO: `/v3/projects/bf55d220c94d4445af9f73c532029e42` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:46:03.389265 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f5f7ec-d18b-4c42-8bcb-788722f1526e admin admin] RBAC: Authorizing `identity:delete_project(project_id=bf55d220c94d4445af9f73c532029e42)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:46:03.391209 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f5f7ec-d18b-4c42-8bcb-788722f1526e admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:46:03.437213 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-53f5f7ec-d18b-4c42-8bcb-788722f1526e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bf55d220c94d4445af9f73c532029e42', 'request_id': 'req-53f5f7ec-d18b-4c42-8bcb-788722f1526e'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:46:03.442544 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-53f5f7ec-d18b-4c42-8bcb-788722f1526e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bf55d220c94d4445af9f73c532029e42', 'request_id': 'req-53f5f7ec-d18b-4c42-8bcb-788722f1526e'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:46:03.448771 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-53f5f7ec-d18b-4c42-8bcb-788722f1526e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bf55d220c94d4445af9f73c532029e42', 'request_id': 'req-53f5f7ec-d18b-4c42-8bcb-788722f1526e'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:46:03.461775 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-53f5f7ec-d18b-4c42-8bcb-788722f1526e admin admin] The token cache is being invalidate because project bf55d220c94d4445af9f73c532029e42 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80311) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 01 16:46:03.462063 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-53f5f7ec-d18b-4c42-8bcb-788722f1526e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:46:03.462760 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-53f5f7ec-d18b-4c42-8bcb-788722f1526e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:46:03.464412 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1195/2408] 10.222.0.57 () {58 vars in 1232 bytes} [Fri Jul 1 16:46:03 2022] DELETE /identity/v3/projects/bf55d220c94d4445af9f73c532029e42 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 16:46:03.475729 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02d6dea-9ec6-4672-8db8-6d69e8320586 None None] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:46:03.476006 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02d6dea-9ec6-4672-8db8-6d69e8320586 None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:46:03.476281 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02d6dea-9ec6-4672-8db8-6d69e8320586 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:46:03.494165 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.auth.core [None req-b02d6dea-9ec6-4672-8db8-6d69e8320586 None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80312) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:46:03.495323 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-b02d6dea-9ec6-4672-8db8-6d69e8320586 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:46:03.553557 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1214/2409] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 1 16:46:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 16:46:03.561331 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6eb98b5-6a07-4e7b-95f5-6b97bb0c0220 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:46:03.561625 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6eb98b5-6a07-4e7b-95f5-6b97bb0c0220 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:46:03.562019 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6eb98b5-6a07-4e7b-95f5-6b97bb0c0220 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:46:03.580166 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-e6eb98b5-6a07-4e7b-95f5-6b97bb0c0220 None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:46:03.581371 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-e6eb98b5-6a07-4e7b-95f5-6b97bb0c0220 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:46:03.590776 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1196/2410] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 1 16:46:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 16:46:03.595125 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02d6dea-9ec6-4672-8db8-6d69e8320586 None None] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:46:03.607142 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02d6dea-9ec6-4672-8db8-6d69e8320586 None None] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:46:03.612445 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db69528-0f61-4687-86b2-1979458f5be7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:46:03.612974 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db69528-0f61-4687-86b2-1979458f5be7 admin admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:46:03.613257 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db69528-0f61-4687-86b2-1979458f5be7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:46:03.613536 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db69528-0f61-4687-86b2-1979458f5be7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:46:03.614411 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db69528-0f61-4687-86b2-1979458f5be7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80312) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:46:03.614730 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db69528-0f61-4687-86b2-1979458f5be7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:46:03.616406 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db69528-0f61-4687-86b2-1979458f5be7 admin admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:46:03.629214 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1215/2411] 10.222.0.57 () {58 vars in 1152 bytes} [Fri Jul 1 16:46:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:46:03.634338 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6eb98b5-6a07-4e7b-95f5-6b97bb0c0220 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:46:03.643951 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6eb98b5-6a07-4e7b-95f5-6b97bb0c0220 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:46:03.649407 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2c7b28-490e-4662-af5d-26da9ad684d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:46:03.650068 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2c7b28-490e-4662-af5d-26da9ad684d4 admin admin] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:46:03.650371 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2c7b28-490e-4662-af5d-26da9ad684d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:46:03.650678 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2c7b28-490e-4662-af5d-26da9ad684d4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:46:03.651401 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a2c7b28-490e-4662-af5d-26da9ad684d4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:46:03.653299 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a2c7b28-490e-4662-af5d-26da9ad684d4 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:46:03.681326 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1197/2412] 10.222.0.57 () {60 vars in 1152 bytes} [Fri Jul 1 16:46:03 2022] POST /identity/v3/projects => generated 367 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 01 16:46:03.685146 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db69528-0f61-4687-86b2-1979458f5be7 admin admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:46:03.691518 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:46:03.696464 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db69528-0f61-4687-86b2-1979458f5be7 admin admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:46:03.761730 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058081cd-8198-4fb5-b157-31227c571b04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:46:03.762330 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058081cd-8198-4fb5-b157-31227c571b04 admin admin] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:46:03.762651 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058081cd-8198-4fb5-b157-31227c571b04 admin admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:46:03.762934 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058081cd-8198-4fb5-b157-31227c571b04 admin admin] PATH_INFO: `/v3/users` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:46:03.763624 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058081cd-8198-4fb5-b157-31227c571b04 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:46:03.765743 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058081cd-8198-4fb5-b157-31227c571b04 admin admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:46:03.786696 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1216/2413] 10.222.0.57 () {60 vars in 1143 bytes} [Fri Jul 1 16:46:03 2022] POST /identity/v3/users => generated 284 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 16:46:03.790905 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2c7b28-490e-4662-af5d-26da9ad684d4 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:46:03.800955 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2c7b28-490e-4662-af5d-26da9ad684d4 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:46:03.805524 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37449ac2-2621-4e9a-9b53-4bba3a3f0b06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:46:03.806183 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37449ac2-2621-4e9a-9b53-4bba3a3f0b06 admin admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:46:03.806494 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37449ac2-2621-4e9a-9b53-4bba3a3f0b06 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:46:03.806772 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37449ac2-2621-4e9a-9b53-4bba3a3f0b06 admin admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:46:03.807494 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37449ac2-2621-4e9a-9b53-4bba3a3f0b06 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:46:03.809077 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37449ac2-2621-4e9a-9b53-4bba3a3f0b06 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:46:03.819811 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1198/2414] 10.222.0.57 () {58 vars in 1121 bytes} [Fri Jul 1 16:46:03 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 16:46:03.823850 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058081cd-8198-4fb5-b157-31227c571b04 admin admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:46:03.834340 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058081cd-8198-4fb5-b157-31227c571b04 admin admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:46:03.839452 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e9568c-6eb8-4752-9ea9-b9ab6baaf3f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:46:03.840108 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e9568c-6eb8-4752-9ea9-b9ab6baaf3f6 admin admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:46:03.840392 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e9568c-6eb8-4752-9ea9-b9ab6baaf3f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:46:03.840666 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e9568c-6eb8-4752-9ea9-b9ab6baaf3f6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:46:03.841422 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e9568c-6eb8-4752-9ea9-b9ab6baaf3f6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:46:03.843080 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e9568c-6eb8-4752-9ea9-b9ab6baaf3f6 admin admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:46:03.853386 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1217/2415] 10.222.0.57 () {58 vars in 1121 bytes} [Fri Jul 1 16:46:03 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 01 16:46:03.857549 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37449ac2-2621-4e9a-9b53-4bba3a3f0b06 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:46:03.868217 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37449ac2-2621-4e9a-9b53-4bba3a3f0b06 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:46:03.872873 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863b58e5-5f8e-466f-bebd-ed0212ab49a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:46:03.873871 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863b58e5-5f8e-466f-bebd-ed0212ab49a5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:46:03.874158 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863b58e5-5f8e-466f-bebd-ed0212ab49a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:46:03.874474 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863b58e5-5f8e-466f-bebd-ed0212ab49a5 admin admin] PATH_INFO: `/v3/projects/4017a901f2b34a83b73ac0e95bf93dbe/users/3fa91c71418b4f63aed389e545041d67/roles/983be40bdab04c478eaa90c6b6a46b14` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:46:03.889817 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863b58e5-5f8e-466f-bebd-ed0212ab49a5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4017a901f2b34a83b73ac0e95bf93dbe, user_id=3fa91c71418b4f63aed389e545041d67, role_id=983be40bdab04c478eaa90c6b6a46b14)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:46:03.892655 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863b58e5-5f8e-466f-bebd-ed0212ab49a5 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:46:03.906312 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1199/2416] 10.222.0.57 () {60 vars in 1482 bytes} [Fri Jul 1 16:46:03 2022] PUT /identity/v3/projects/4017a901f2b34a83b73ac0e95bf93dbe/users/3fa91c71418b4f63aed389e545041d67/roles/983be40bdab04c478eaa90c6b6a46b14 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 16:46:03.918091 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3392a5-3550-4352-9b19-97ccc8f82971 None None] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:46:03.918410 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3392a5-3550-4352-9b19-97ccc8f82971 None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:46:03.918713 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3392a5-3550-4352-9b19-97ccc8f82971 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:46:03.945525 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.auth.core [None req-1d3392a5-3550-4352-9b19-97ccc8f82971 None None] MFA Rules not processed for user `3fa91c71418b4f63aed389e545041d67`. Rule list: `[]` (Enabled: `True`). {{(pid=80312) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:46:03.946760 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-1d3392a5-3550-4352-9b19-97ccc8f82971 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:46:04.011349 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1218/2417] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 1 16:46:03 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 16:46:04.022225 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863b58e5-5f8e-466f-bebd-ed0212ab49a5 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:46:04.023975 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-863b58e5-5f8e-466f-bebd-ed0212ab49a5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:46:04.029357 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:46:04.082535 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863b58e5-5f8e-466f-bebd-ed0212ab49a5 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:46:04.086634 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddff6669-d6be-4c3c-b659-44be81fb2e40 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbddf971a2354bbdbb9da1d63f3abf88', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0788e6725a84e57a4174284c1845755', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:46:04.087277 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddff6669-d6be-4c3c-b659-44be81fb2e40 service cinder] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:46:04.087563 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddff6669-d6be-4c3c-b659-44be81fb2e40 service cinder] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:46:04.087842 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddff6669-d6be-4c3c-b659-44be81fb2e40 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:46:04.096033 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddff6669-d6be-4c3c-b659-44be81fb2e40 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:46:04.097922 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddff6669-d6be-4c3c-b659-44be81fb2e40 service cinder] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:46:04.106852 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1200/2418] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 1 16:46:04 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 16:46:12.572593 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3392a5-3550-4352-9b19-97ccc8f82971 None None] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:46:12.635518 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3392a5-3550-4352-9b19-97ccc8f82971 None None] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:46:12.640047 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30c3ec4-a25e-42da-91a2-8c72dba9521e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:46:12.640927 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30c3ec4-a25e-42da-91a2-8c72dba9521e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:46:12.641215 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30c3ec4-a25e-42da-91a2-8c72dba9521e admin admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:46:12.641492 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30c3ec4-a25e-42da-91a2-8c72dba9521e admin admin] PATH_INFO: `/v3/users/3fa91c71418b4f63aed389e545041d67` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:46:12.642692 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c30c3ec4-a25e-42da-91a2-8c72dba9521e admin admin] RBAC: Authorizing `identity:delete_user(user_id=3fa91c71418b4f63aed389e545041d67)` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:46:12.644616 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c30c3ec4-a25e-42da-91a2-8c72dba9521e admin admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:46:12.658905 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=96,keystone:INSERT=3,keystone:DELETE=1 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:46:12.695786 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-c30c3ec4-a25e-42da-91a2-8c72dba9521e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3fa91c71418b4f63aed389e545041d67'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:46:12.711883 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-c30c3ec4-a25e-42da-91a2-8c72dba9521e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3fa91c71418b4f63aed389e545041d67'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:46:12.719073 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-c30c3ec4-a25e-42da-91a2-8c72dba9521e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3fa91c71418b4f63aed389e545041d67'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:46:12.719795 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-c30c3ec4-a25e-42da-91a2-8c72dba9521e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3fa91c71418b4f63aed389e545041d67'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:46:12.720391 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-c30c3ec4-a25e-42da-91a2-8c72dba9521e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3fa91c71418b4f63aed389e545041d67'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:46:12.734734 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1219/2419] 10.222.0.57 () {58 vars in 1223 bytes} [Fri Jul 1 16:46:12 2022] DELETE /identity/v3/users/3fa91c71418b4f63aed389e545041d67 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 16:46:12.739175 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada8f6da-1261-4d52-91cc-166eedbd8b17 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:46:12.739640 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada8f6da-1261-4d52-91cc-166eedbd8b17 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:46:12.739839 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada8f6da-1261-4d52-91cc-166eedbd8b17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:46:12.757376 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-ada8f6da-1261-4d52-91cc-166eedbd8b17 None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:46:12.758634 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-ada8f6da-1261-4d52-91cc-166eedbd8b17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:46:12.817257 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1201/2420] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 1 16:46:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 16:46:12.826990 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30c3ec4-a25e-42da-91a2-8c72dba9521e admin admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:46:12.828417 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-c30c3ec4-a25e-42da-91a2-8c72dba9521e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:46:12.895364 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30c3ec4-a25e-42da-91a2-8c72dba9521e admin admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:46:12.900715 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ced79ea-944a-48ec-90b8-dc038dde88bc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ed200ce29684c5f80354fa9a8638fe2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0788e6725a84e57a4174284c1845755', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:46:12.901376 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ced79ea-944a-48ec-90b8-dc038dde88bc service neutron] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:46:12.901664 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ced79ea-944a-48ec-90b8-dc038dde88bc service neutron] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:46:12.901959 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ced79ea-944a-48ec-90b8-dc038dde88bc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:46:12.912308 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ced79ea-944a-48ec-90b8-dc038dde88bc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:46:12.913834 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ced79ea-944a-48ec-90b8-dc038dde88bc service neutron] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:46:12.923394 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1220/2421] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 1 16:46:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 16:46:13.204498 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada8f6da-1261-4d52-91cc-166eedbd8b17 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:46:13.206204 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-ada8f6da-1261-4d52-91cc-166eedbd8b17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:46:13.218783 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada8f6da-1261-4d52-91cc-166eedbd8b17 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:46:13.223468 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9942529-3bfa-43cf-9981-e3ee0c1ea45b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:46:13.225023 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9942529-3bfa-43cf-9981-e3ee0c1ea45b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:46:13.225311 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9942529-3bfa-43cf-9981-e3ee0c1ea45b admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:46:13.225594 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9942529-3bfa-43cf-9981-e3ee0c1ea45b admin admin] PATH_INFO: `/v3/projects/4017a901f2b34a83b73ac0e95bf93dbe` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:46:13.227326 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9942529-3bfa-43cf-9981-e3ee0c1ea45b admin admin] RBAC: Authorizing `identity:delete_project(project_id=4017a901f2b34a83b73ac0e95bf93dbe)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:46:13.228790 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9942529-3bfa-43cf-9981-e3ee0c1ea45b admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:46:13.278259 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-f9942529-3bfa-43cf-9981-e3ee0c1ea45b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4017a901f2b34a83b73ac0e95bf93dbe', 'request_id': 'req-f9942529-3bfa-43cf-9981-e3ee0c1ea45b'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:46:13.283904 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-f9942529-3bfa-43cf-9981-e3ee0c1ea45b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4017a901f2b34a83b73ac0e95bf93dbe', 'request_id': 'req-f9942529-3bfa-43cf-9981-e3ee0c1ea45b'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:46:13.290904 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-f9942529-3bfa-43cf-9981-e3ee0c1ea45b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4017a901f2b34a83b73ac0e95bf93dbe', 'request_id': 'req-f9942529-3bfa-43cf-9981-e3ee0c1ea45b'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:46:13.302779 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-f9942529-3bfa-43cf-9981-e3ee0c1ea45b admin admin] The token cache is being invalidate because project 4017a901f2b34a83b73ac0e95bf93dbe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80311) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 01 16:46:13.303070 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-f9942529-3bfa-43cf-9981-e3ee0c1ea45b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:46:13.303716 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-f9942529-3bfa-43cf-9981-e3ee0c1ea45b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:46:13.305258 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1202/2422] 10.222.0.57 () {58 vars in 1232 bytes} [Fri Jul 1 16:46:13 2022] DELETE /identity/v3/projects/4017a901f2b34a83b73ac0e95bf93dbe => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 16:46:13.310158 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2bc3eb-221b-452b-8181-db3d48e2f900 None None] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:46:13.310484 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2bc3eb-221b-452b-8181-db3d48e2f900 None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:46:13.310769 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2bc3eb-221b-452b-8181-db3d48e2f900 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:46:13.329182 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.auth.core [None req-6e2bc3eb-221b-452b-8181-db3d48e2f900 None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80312) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:46:13.330419 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-6e2bc3eb-221b-452b-8181-db3d48e2f900 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:46:13.389597 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1221/2423] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 1 16:46:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 16:46:13.398283 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1883764e-f524-429b-b001-b470ab5e749a None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:46:13.398604 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1883764e-f524-429b-b001-b470ab5e749a None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:46:13.398884 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1883764e-f524-429b-b001-b470ab5e749a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:46:13.417147 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-1883764e-f524-429b-b001-b470ab5e749a None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:46:13.418336 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-1883764e-f524-429b-b001-b470ab5e749a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:46:13.426067 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1203/2424] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 1 16:46:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 01 16:46:13.430902 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2bc3eb-221b-452b-8181-db3d48e2f900 None None] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:46:13.440626 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2bc3eb-221b-452b-8181-db3d48e2f900 None None] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:46:13.446012 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d412db11-01bc-4336-b91c-9d911f5dd463 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:46:13.446706 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d412db11-01bc-4336-b91c-9d911f5dd463 admin admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:46:13.446996 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d412db11-01bc-4336-b91c-9d911f5dd463 admin admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:46:13.447277 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d412db11-01bc-4336-b91c-9d911f5dd463 admin admin] PATH_INFO: `/v3/domains` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:46:13.448009 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d412db11-01bc-4336-b91c-9d911f5dd463 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80312) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:46:13.448317 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d412db11-01bc-4336-b91c-9d911f5dd463 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:46:13.449923 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d412db11-01bc-4336-b91c-9d911f5dd463 admin admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:46:13.462272 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1222/2425] 10.222.0.57 () {58 vars in 1152 bytes} [Fri Jul 1 16:46:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:46:13.466822 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1883764e-f524-429b-b001-b470ab5e749a None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:46:13.476148 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1883764e-f524-429b-b001-b470ab5e749a None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:46:13.480784 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623916af-3228-4a5f-b5ac-2e79684fb39b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:46:13.481469 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623916af-3228-4a5f-b5ac-2e79684fb39b admin admin] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:46:13.481755 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623916af-3228-4a5f-b5ac-2e79684fb39b admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:46:13.482032 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623916af-3228-4a5f-b5ac-2e79684fb39b admin admin] PATH_INFO: `/v3/projects` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:46:13.482748 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-623916af-3228-4a5f-b5ac-2e79684fb39b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:46:13.484699 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-623916af-3228-4a5f-b5ac-2e79684fb39b admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:46:13.504738 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:46:13.514376 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1204/2426] 10.222.0.57 () {60 vars in 1152 bytes} [Fri Jul 1 16:46:13 2022] POST /identity/v3/projects => generated 379 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 16:46:13.517650 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d412db11-01bc-4336-b91c-9d911f5dd463 admin admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:46:13.527616 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d412db11-01bc-4336-b91c-9d911f5dd463 admin admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:46:13.546960 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:46:13.588757 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4f351b-760f-4fd9-b280-a2cc3b7596db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:46:13.589459 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4f351b-760f-4fd9-b280-a2cc3b7596db admin admin] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:46:13.589747 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4f351b-760f-4fd9-b280-a2cc3b7596db admin admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:46:13.590024 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4f351b-760f-4fd9-b280-a2cc3b7596db admin admin] PATH_INFO: `/v3/users` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:46:13.590750 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c4f351b-760f-4fd9-b280-a2cc3b7596db admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:46:13.592706 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c4f351b-760f-4fd9-b280-a2cc3b7596db admin admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:46:13.621100 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1223/2427] 10.222.0.57 () {60 vars in 1143 bytes} [Fri Jul 1 16:46:13 2022] POST /identity/v3/users => generated 290 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 16:46:13.622374 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623916af-3228-4a5f-b5ac-2e79684fb39b admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:46:13.635180 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623916af-3228-4a5f-b5ac-2e79684fb39b admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:46:13.640067 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb0d8af-d7d8-43e0-90e4-ea9380082ff6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:46:13.641027 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb0d8af-d7d8-43e0-90e4-ea9380082ff6 admin admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:46:13.641467 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb0d8af-d7d8-43e0-90e4-ea9380082ff6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:46:13.641753 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb0d8af-d7d8-43e0-90e4-ea9380082ff6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:46:13.642547 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb0d8af-d7d8-43e0-90e4-ea9380082ff6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:46:13.644155 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb0d8af-d7d8-43e0-90e4-ea9380082ff6 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:46:13.655633 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1205/2428] 10.222.0.57 () {58 vars in 1121 bytes} [Fri Jul 1 16:46:13 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 16:46:13.659830 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4f351b-760f-4fd9-b280-a2cc3b7596db admin admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:46:13.670056 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4f351b-760f-4fd9-b280-a2cc3b7596db admin admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:46:13.674871 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881462f7-38b3-48a1-bb37-4741befd1f63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:46:13.675635 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881462f7-38b3-48a1-bb37-4741befd1f63 admin admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:46:13.676023 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881462f7-38b3-48a1-bb37-4741befd1f63 admin admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:46:13.676400 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881462f7-38b3-48a1-bb37-4741befd1f63 admin admin] PATH_INFO: `/v3/roles` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:46:13.677764 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-881462f7-38b3-48a1-bb37-4741befd1f63 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:46:13.679566 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-881462f7-38b3-48a1-bb37-4741befd1f63 admin admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:46:13.690715 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1224/2429] 10.222.0.57 () {58 vars in 1121 bytes} [Fri Jul 1 16:46:13 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 16:46:13.695589 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb0d8af-d7d8-43e0-90e4-ea9380082ff6 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:46:13.705637 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb0d8af-d7d8-43e0-90e4-ea9380082ff6 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:46:13.711776 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a17c99f-bfb8-432f-914a-3f73ac621e0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:46:13.712949 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a17c99f-bfb8-432f-914a-3f73ac621e0f admin admin] REQUEST_METHOD: `PUT` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:46:13.713374 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a17c99f-bfb8-432f-914a-3f73ac621e0f admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:46:13.713713 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a17c99f-bfb8-432f-914a-3f73ac621e0f admin admin] PATH_INFO: `/v3/projects/f6c0a787b5c44452860dd16503933a35/users/e091e66e78e34cc7bf2f265940ed01bd/roles/983be40bdab04c478eaa90c6b6a46b14` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:46:13.729942 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a17c99f-bfb8-432f-914a-3f73ac621e0f admin admin] RBAC: Authorizing `identity:create_grant(project_id=f6c0a787b5c44452860dd16503933a35, user_id=e091e66e78e34cc7bf2f265940ed01bd, role_id=983be40bdab04c478eaa90c6b6a46b14)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:46:13.732617 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a17c99f-bfb8-432f-914a-3f73ac621e0f admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:46:13.751608 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1206/2430] 10.222.0.57 () {60 vars in 1482 bytes} [Fri Jul 1 16:46:13 2022] PUT /identity/v3/projects/f6c0a787b5c44452860dd16503933a35/users/e091e66e78e34cc7bf2f265940ed01bd/roles/983be40bdab04c478eaa90c6b6a46b14 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 01 16:46:13.763396 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ed5912-be4d-43a5-bbd9-3ee5afca78d6 None None] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:46:13.763687 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ed5912-be4d-43a5-bbd9-3ee5afca78d6 None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:46:13.763994 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ed5912-be4d-43a5-bbd9-3ee5afca78d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:46:13.792189 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.auth.core [None req-06ed5912-be4d-43a5-bbd9-3ee5afca78d6 None None] MFA Rules not processed for user `e091e66e78e34cc7bf2f265940ed01bd`. Rule list: `[]` (Enabled: `True`). {{(pid=80312) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:46:13.793415 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-06ed5912-be4d-43a5-bbd9-3ee5afca78d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:46:13.859167 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1225/2431] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 1 16:46:13 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 16:46:13.863912 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a17c99f-bfb8-432f-914a-3f73ac621e0f admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:46:13.928417 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a17c99f-bfb8-432f-914a-3f73ac621e0f admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:46:13.933154 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd842c44-51a7-47b0-a965-1dfbd62c9d77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:46:13.933814 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd842c44-51a7-47b0-a965-1dfbd62c9d77 admin admin] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:46:13.934102 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd842c44-51a7-47b0-a965-1dfbd62c9d77 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:46:13.934419 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd842c44-51a7-47b0-a965-1dfbd62c9d77 admin admin] PATH_INFO: `/v3/projects` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:46:13.935101 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd842c44-51a7-47b0-a965-1dfbd62c9d77 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:46:13.937013 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd842c44-51a7-47b0-a965-1dfbd62c9d77 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:46:13.962256 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1207/2432] 10.222.0.57 () {60 vars in 1152 bytes} [Fri Jul 1 16:46:13 2022] POST /identity/v3/projects => generated 381 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 16:46:13.966374 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ed5912-be4d-43a5-bbd9-3ee5afca78d6 None None] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:46:13.978286 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ed5912-be4d-43a5-bbd9-3ee5afca78d6 None None] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:46:14.038325 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e936b75a-dda9-49e9-b4e6-e01068814ebc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:46:14.039027 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e936b75a-dda9-49e9-b4e6-e01068814ebc admin admin] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:46:14.039412 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e936b75a-dda9-49e9-b4e6-e01068814ebc admin admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:46:14.039699 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e936b75a-dda9-49e9-b4e6-e01068814ebc admin admin] PATH_INFO: `/v3/users` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:46:14.040386 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e936b75a-dda9-49e9-b4e6-e01068814ebc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:46:14.042397 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e936b75a-dda9-49e9-b4e6-e01068814ebc admin admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:46:14.065122 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1226/2433] 10.222.0.57 () {60 vars in 1143 bytes} [Fri Jul 1 16:46:13 2022] POST /identity/v3/users => generated 297 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 16:46:14.066409 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd842c44-51a7-47b0-a965-1dfbd62c9d77 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:46:14.076697 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd842c44-51a7-47b0-a965-1dfbd62c9d77 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:46:14.081669 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7797c5ba-1d9b-4fea-8d2f-6835264f16f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:46:14.082324 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7797c5ba-1d9b-4fea-8d2f-6835264f16f2 admin admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:46:14.082636 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7797c5ba-1d9b-4fea-8d2f-6835264f16f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:46:14.082913 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7797c5ba-1d9b-4fea-8d2f-6835264f16f2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:46:14.084506 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7797c5ba-1d9b-4fea-8d2f-6835264f16f2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:46:14.086094 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7797c5ba-1d9b-4fea-8d2f-6835264f16f2 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:46:14.096642 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1208/2434] 10.222.0.57 () {58 vars in 1121 bytes} [Fri Jul 1 16:46:14 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 16:46:14.101872 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e936b75a-dda9-49e9-b4e6-e01068814ebc admin admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:46:14.111779 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e936b75a-dda9-49e9-b4e6-e01068814ebc admin admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:46:14.116739 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30bca99-7495-4401-a47a-417ca3098a33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:46:14.117608 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30bca99-7495-4401-a47a-417ca3098a33 admin admin] REQUEST_METHOD: `PUT` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:46:14.117891 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30bca99-7495-4401-a47a-417ca3098a33 admin admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:46:14.118170 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30bca99-7495-4401-a47a-417ca3098a33 admin admin] PATH_INFO: `/v3/projects/0931d09302d94f90b0a2ad7d10329112/users/676a2c5e213748e5853836c7d289af69/roles/84ee739146934c1ab32c501ff8093644` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:46:14.132894 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b30bca99-7495-4401-a47a-417ca3098a33 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0931d09302d94f90b0a2ad7d10329112, user_id=676a2c5e213748e5853836c7d289af69, role_id=84ee739146934c1ab32c501ff8093644)` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:46:14.135656 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b30bca99-7495-4401-a47a-417ca3098a33 admin admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:46:14.138801 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:46:14.150437 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1227/2435] 10.222.0.57 () {60 vars in 1482 bytes} [Fri Jul 1 16:46:14 2022] PUT /identity/v3/projects/0931d09302d94f90b0a2ad7d10329112/users/676a2c5e213748e5853836c7d289af69/roles/84ee739146934c1ab32c501ff8093644 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 16:46:14.154760 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7797c5ba-1d9b-4fea-8d2f-6835264f16f2 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:46:14.165626 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7797c5ba-1d9b-4fea-8d2f-6835264f16f2 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:46:14.220823 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822e4ad8-f537-4879-bdf0-dd42ea265f1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:46:14.221486 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822e4ad8-f537-4879-bdf0-dd42ea265f1b admin admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:46:14.221773 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822e4ad8-f537-4879-bdf0-dd42ea265f1b admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:46:14.222056 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822e4ad8-f537-4879-bdf0-dd42ea265f1b admin admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:46:14.222813 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-822e4ad8-f537-4879-bdf0-dd42ea265f1b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:46:14.224975 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-822e4ad8-f537-4879-bdf0-dd42ea265f1b admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:46:14.234702 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1209/2436] 10.222.0.57 () {58 vars in 1121 bytes} [Fri Jul 1 16:46:14 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 16:46:14.239038 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30bca99-7495-4401-a47a-417ca3098a33 admin admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:46:14.249599 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30bca99-7495-4401-a47a-417ca3098a33 admin admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:46:14.254103 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0c2eaf-8859-46ea-8325-37baa2e1d839 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:46:14.255016 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0c2eaf-8859-46ea-8325-37baa2e1d839 admin admin] REQUEST_METHOD: `PUT` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:46:14.255775 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0c2eaf-8859-46ea-8325-37baa2e1d839 admin admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:46:14.256060 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0c2eaf-8859-46ea-8325-37baa2e1d839 admin admin] PATH_INFO: `/v3/projects/0931d09302d94f90b0a2ad7d10329112/users/676a2c5e213748e5853836c7d289af69/roles/84ee739146934c1ab32c501ff8093644` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:46:14.258269 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d0c2eaf-8859-46ea-8325-37baa2e1d839 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0931d09302d94f90b0a2ad7d10329112, user_id=676a2c5e213748e5853836c7d289af69, role_id=84ee739146934c1ab32c501ff8093644)` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:46:14.261232 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d0c2eaf-8859-46ea-8325-37baa2e1d839 admin admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:46:14.272574 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1228/2437] 10.222.0.57 () {60 vars in 1482 bytes} [Fri Jul 1 16:46:14 2022] PUT /identity/v3/projects/0931d09302d94f90b0a2ad7d10329112/users/676a2c5e213748e5853836c7d289af69/roles/84ee739146934c1ab32c501ff8093644 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 16:46:14.283092 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2915c9-1345-48a7-8e8e-f68a55075f82 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:46:14.283387 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2915c9-1345-48a7-8e8e-f68a55075f82 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:46:14.283672 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2915c9-1345-48a7-8e8e-f68a55075f82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:46:14.310179 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-8a2915c9-1345-48a7-8e8e-f68a55075f82 None None] MFA Rules not processed for user `676a2c5e213748e5853836c7d289af69`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:46:14.311423 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-8a2915c9-1345-48a7-8e8e-f68a55075f82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:46:14.336024 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:46:14.384123 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1210/2438] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 1 16:46:14 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 01 16:46:14.396661 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0c2eaf-8859-46ea-8325-37baa2e1d839 admin admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:46:14.398463 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-1d0c2eaf-8859-46ea-8325-37baa2e1d839 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:46:14.456278 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0c2eaf-8859-46ea-8325-37baa2e1d839 admin admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:46:14.461845 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606954e8-5fce-423e-afca-1051b1094bf8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbddf971a2354bbdbb9da1d63f3abf88', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0788e6725a84e57a4174284c1845755', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:46:14.462548 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606954e8-5fce-423e-afca-1051b1094bf8 service cinder] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:46:14.462833 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606954e8-5fce-423e-afca-1051b1094bf8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:46:14.463109 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606954e8-5fce-423e-afca-1051b1094bf8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:46:14.472127 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-606954e8-5fce-423e-afca-1051b1094bf8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:46:14.473560 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-606954e8-5fce-423e-afca-1051b1094bf8 service cinder] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:46:14.484049 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1229/2439] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 1 16:46:14 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 16:46:14.790402 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2915c9-1345-48a7-8e8e-f68a55075f82 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:46:14.799365 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2915c9-1345-48a7-8e8e-f68a55075f82 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:46:14.804213 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2154930-e884-4ff1-ab1e-e41c83861821 tempest-VolumeQuotaClassesTest-1014181776 tempest-VolumeQuotaClassesTest-1014181776-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '676a2c5e213748e5853836c7d289af69', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0931d09302d94f90b0a2ad7d10329112', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:46:14.804882 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2154930-e884-4ff1-ab1e-e41c83861821 tempest-VolumeQuotaClassesTest-1014181776 tempest-VolumeQuotaClassesTest-1014181776-project-admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:46:14.805174 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2154930-e884-4ff1-ab1e-e41c83861821 tempest-VolumeQuotaClassesTest-1014181776 tempest-VolumeQuotaClassesTest-1014181776-project-admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:46:14.805463 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2154930-e884-4ff1-ab1e-e41c83861821 tempest-VolumeQuotaClassesTest-1014181776 tempest-VolumeQuotaClassesTest-1014181776-project-admin] PATH_INFO: `/v3/domains` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:46:14.806216 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2154930-e884-4ff1-ab1e-e41c83861821 tempest-VolumeQuotaClassesTest-1014181776 tempest-VolumeQuotaClassesTest-1014181776-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:46:14.806564 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2154930-e884-4ff1-ab1e-e41c83861821 tempest-VolumeQuotaClassesTest-1014181776 tempest-VolumeQuotaClassesTest-1014181776-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:46:14.808163 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2154930-e884-4ff1-ab1e-e41c83861821 tempest-VolumeQuotaClassesTest-1014181776 tempest-VolumeQuotaClassesTest-1014181776-project-admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:46:14.822209 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1211/2440] 10.222.0.57 () {58 vars in 1152 bytes} [Fri Jul 1 16:46:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:46:14.826145 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606954e8-5fce-423e-afca-1051b1094bf8 service cinder] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:46:14.838424 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606954e8-5fce-423e-afca-1051b1094bf8 service cinder] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:46:14.842556 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fbf100a-080c-4ba0-aba9-fc46a1c323b8 tempest-VolumeQuotaClassesTest-1014181776 tempest-VolumeQuotaClassesTest-1014181776-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '676a2c5e213748e5853836c7d289af69', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0931d09302d94f90b0a2ad7d10329112', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:46:14.843230 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbf100a-080c-4ba0-aba9-fc46a1c323b8 tempest-VolumeQuotaClassesTest-1014181776 tempest-VolumeQuotaClassesTest-1014181776-project-admin] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:46:14.844044 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbf100a-080c-4ba0-aba9-fc46a1c323b8 tempest-VolumeQuotaClassesTest-1014181776 tempest-VolumeQuotaClassesTest-1014181776-project-admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:46:14.844044 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbf100a-080c-4ba0-aba9-fc46a1c323b8 tempest-VolumeQuotaClassesTest-1014181776 tempest-VolumeQuotaClassesTest-1014181776-project-admin] PATH_INFO: `/v3/projects` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:46:14.844484 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fbf100a-080c-4ba0-aba9-fc46a1c323b8 tempest-VolumeQuotaClassesTest-1014181776 tempest-VolumeQuotaClassesTest-1014181776-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:46:14.846431 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fbf100a-080c-4ba0-aba9-fc46a1c323b8 tempest-VolumeQuotaClassesTest-1014181776 tempest-VolumeQuotaClassesTest-1014181776-project-admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:46:14.878309 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1230/2441] 10.222.0.57 () {60 vars in 1152 bytes} [Fri Jul 1 16:46:14 2022] POST /identity/v3/projects => generated 355 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 16:46:14.879612 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2154930-e884-4ff1-ab1e-e41c83861821 tempest-VolumeQuotaClassesTest-1014181776 tempest-VolumeQuotaClassesTest-1014181776-project-admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:46:14.891325 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2154930-e884-4ff1-ab1e-e41c83861821 tempest-VolumeQuotaClassesTest-1014181776 tempest-VolumeQuotaClassesTest-1014181776-project-admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:46:14.949994 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2b21c6-5186-455c-9c4b-f8eb2f4fd554 tempest-VolumeQuotaClassesTest-1014181776 tempest-VolumeQuotaClassesTest-1014181776-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '676a2c5e213748e5853836c7d289af69', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0931d09302d94f90b0a2ad7d10329112', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:46:14.950691 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2b21c6-5186-455c-9c4b-f8eb2f4fd554 tempest-VolumeQuotaClassesTest-1014181776 tempest-VolumeQuotaClassesTest-1014181776-project-admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:46:14.950979 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2b21c6-5186-455c-9c4b-f8eb2f4fd554 tempest-VolumeQuotaClassesTest-1014181776 tempest-VolumeQuotaClassesTest-1014181776-project-admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:46:14.951260 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2b21c6-5186-455c-9c4b-f8eb2f4fd554 tempest-VolumeQuotaClassesTest-1014181776 tempest-VolumeQuotaClassesTest-1014181776-project-admin] PATH_INFO: `/v3/domains` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:46:14.951997 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de2b21c6-5186-455c-9c4b-f8eb2f4fd554 tempest-VolumeQuotaClassesTest-1014181776 tempest-VolumeQuotaClassesTest-1014181776-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:46:14.952306 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de2b21c6-5186-455c-9c4b-f8eb2f4fd554 tempest-VolumeQuotaClassesTest-1014181776 tempest-VolumeQuotaClassesTest-1014181776-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:46:14.953895 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de2b21c6-5186-455c-9c4b-f8eb2f4fd554 tempest-VolumeQuotaClassesTest-1014181776 tempest-VolumeQuotaClassesTest-1014181776-project-admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:46:14.965440 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1212/2442] 10.222.0.57 () {58 vars in 1152 bytes} [Fri Jul 1 16:46:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:46:15.002168 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fbf100a-080c-4ba0-aba9-fc46a1c323b8 tempest-VolumeQuotaClassesTest-1014181776 tempest-VolumeQuotaClassesTest-1014181776-project-admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:46:15.013322 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fbf100a-080c-4ba0-aba9-fc46a1c323b8 tempest-VolumeQuotaClassesTest-1014181776 tempest-VolumeQuotaClassesTest-1014181776-project-admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:46:15.017870 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179f4b37-f5f0-4fba-8c9e-1c4b8b2e3b9b tempest-VolumeQuotaClassesTest-1014181776 tempest-VolumeQuotaClassesTest-1014181776-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '676a2c5e213748e5853836c7d289af69', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0931d09302d94f90b0a2ad7d10329112', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:46:15.018753 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179f4b37-f5f0-4fba-8c9e-1c4b8b2e3b9b tempest-VolumeQuotaClassesTest-1014181776 tempest-VolumeQuotaClassesTest-1014181776-project-admin] REQUEST_METHOD: `DELETE` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:46:15.019047 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179f4b37-f5f0-4fba-8c9e-1c4b8b2e3b9b tempest-VolumeQuotaClassesTest-1014181776 tempest-VolumeQuotaClassesTest-1014181776-project-admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:46:15.019325 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179f4b37-f5f0-4fba-8c9e-1c4b8b2e3b9b tempest-VolumeQuotaClassesTest-1014181776 tempest-VolumeQuotaClassesTest-1014181776-project-admin] PATH_INFO: `/v3/projects/ce8c7386b9674aa98cbca269bfc584ed` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:46:15.020505 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-179f4b37-f5f0-4fba-8c9e-1c4b8b2e3b9b tempest-VolumeQuotaClassesTest-1014181776 tempest-VolumeQuotaClassesTest-1014181776-project-admin] RBAC: Authorizing `identity:delete_project(project_id=ce8c7386b9674aa98cbca269bfc584ed)` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:46:15.022607 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-179f4b37-f5f0-4fba-8c9e-1c4b8b2e3b9b tempest-VolumeQuotaClassesTest-1014181776 tempest-VolumeQuotaClassesTest-1014181776-project-admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:46:15.067596 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-179f4b37-f5f0-4fba-8c9e-1c4b8b2e3b9b tempest-VolumeQuotaClassesTest-1014181776 tempest-VolumeQuotaClassesTest-1014181776-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ce8c7386b9674aa98cbca269bfc584ed', 'request_id': 'req-179f4b37-f5f0-4fba-8c9e-1c4b8b2e3b9b'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:46:15.075301 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-179f4b37-f5f0-4fba-8c9e-1c4b8b2e3b9b tempest-VolumeQuotaClassesTest-1014181776 tempest-VolumeQuotaClassesTest-1014181776-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ce8c7386b9674aa98cbca269bfc584ed', 'request_id': 'req-179f4b37-f5f0-4fba-8c9e-1c4b8b2e3b9b'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:46:15.080996 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-179f4b37-f5f0-4fba-8c9e-1c4b8b2e3b9b tempest-VolumeQuotaClassesTest-1014181776 tempest-VolumeQuotaClassesTest-1014181776-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ce8c7386b9674aa98cbca269bfc584ed', 'request_id': 'req-179f4b37-f5f0-4fba-8c9e-1c4b8b2e3b9b'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:46:15.092907 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-179f4b37-f5f0-4fba-8c9e-1c4b8b2e3b9b tempest-VolumeQuotaClassesTest-1014181776 tempest-VolumeQuotaClassesTest-1014181776-project-admin] The token cache is being invalidate because project ce8c7386b9674aa98cbca269bfc584ed was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80312) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 01 16:46:15.093193 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-179f4b37-f5f0-4fba-8c9e-1c4b8b2e3b9b tempest-VolumeQuotaClassesTest-1014181776 tempest-VolumeQuotaClassesTest-1014181776-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:46:15.093868 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-179f4b37-f5f0-4fba-8c9e-1c4b8b2e3b9b tempest-VolumeQuotaClassesTest-1014181776 tempest-VolumeQuotaClassesTest-1014181776-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:46:15.095445 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1231/2443] 10.222.0.57 () {58 vars in 1232 bytes} [Fri Jul 1 16:46:15 2022] DELETE /identity/v3/projects/ce8c7386b9674aa98cbca269bfc584ed => generated 0 bytes in 94 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 16:46:15.284571 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2b21c6-5186-455c-9c4b-f8eb2f4fd554 tempest-VolumeQuotaClassesTest-1014181776 tempest-VolumeQuotaClassesTest-1014181776-project-admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:46:15.286289 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-de2b21c6-5186-455c-9c4b-f8eb2f4fd554 tempest-VolumeQuotaClassesTest-1014181776 tempest-VolumeQuotaClassesTest-1014181776-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:46:15.370534 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2b21c6-5186-455c-9c4b-f8eb2f4fd554 tempest-VolumeQuotaClassesTest-1014181776 tempest-VolumeQuotaClassesTest-1014181776-project-admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:46:15.417670 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:46:15.429366 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d01d5ff-2dfd-4b6a-bb7b-fa21d687706a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:46:15.430235 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d01d5ff-2dfd-4b6a-bb7b-fa21d687706a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:46:15.430545 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d01d5ff-2dfd-4b6a-bb7b-fa21d687706a admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:46:15.430823 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d01d5ff-2dfd-4b6a-bb7b-fa21d687706a admin admin] PATH_INFO: `/v3/users/e091e66e78e34cc7bf2f265940ed01bd` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:46:15.432001 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d01d5ff-2dfd-4b6a-bb7b-fa21d687706a admin admin] RBAC: Authorizing `identity:delete_user(user_id=e091e66e78e34cc7bf2f265940ed01bd)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:46:15.434073 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d01d5ff-2dfd-4b6a-bb7b-fa21d687706a admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:46:15.481185 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-5d01d5ff-2dfd-4b6a-bb7b-fa21d687706a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e091e66e78e34cc7bf2f265940ed01bd'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:46:15.481847 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-5d01d5ff-2dfd-4b6a-bb7b-fa21d687706a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e091e66e78e34cc7bf2f265940ed01bd'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:46:15.482493 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-5d01d5ff-2dfd-4b6a-bb7b-fa21d687706a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e091e66e78e34cc7bf2f265940ed01bd'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:46:15.490588 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-5d01d5ff-2dfd-4b6a-bb7b-fa21d687706a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e091e66e78e34cc7bf2f265940ed01bd'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:46:15.503076 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-5d01d5ff-2dfd-4b6a-bb7b-fa21d687706a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e091e66e78e34cc7bf2f265940ed01bd'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:46:15.515820 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1213/2444] 10.222.0.57 () {58 vars in 1223 bytes} [Fri Jul 1 16:46:15 2022] DELETE /identity/v3/users/e091e66e78e34cc7bf2f265940ed01bd => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 16:46:15.520454 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e601d398-dc6d-4846-b579-b590584307e8 None None] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:46:15.520752 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e601d398-dc6d-4846-b579-b590584307e8 None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:46:15.521742 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e601d398-dc6d-4846-b579-b590584307e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:46:15.539690 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.auth.core [None req-e601d398-dc6d-4846-b579-b590584307e8 None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80312) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:46:15.540905 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-e601d398-dc6d-4846-b579-b590584307e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:46:15.582471 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:46:15.603733 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1232/2445] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 1 16:46:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 16:46:15.613228 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d01d5ff-2dfd-4b6a-bb7b-fa21d687706a admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:46:15.614736 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-5d01d5ff-2dfd-4b6a-bb7b-fa21d687706a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:46:15.673412 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d01d5ff-2dfd-4b6a-bb7b-fa21d687706a admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:46:15.677559 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3054034c-e1c3-4125-a527-aad809aa86aa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ed200ce29684c5f80354fa9a8638fe2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0788e6725a84e57a4174284c1845755', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:46:15.678323 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3054034c-e1c3-4125-a527-aad809aa86aa service neutron] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:46:15.678643 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3054034c-e1c3-4125-a527-aad809aa86aa service neutron] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:46:15.678970 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3054034c-e1c3-4125-a527-aad809aa86aa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:46:15.689161 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3054034c-e1c3-4125-a527-aad809aa86aa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:46:15.690800 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3054034c-e1c3-4125-a527-aad809aa86aa service neutron] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:46:15.703341 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1214/2446] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 1 16:46:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 16:46:15.993834 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e601d398-dc6d-4846-b579-b590584307e8 None None] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:46:15.995433 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-e601d398-dc6d-4846-b579-b590584307e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:46:16.008920 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e601d398-dc6d-4846-b579-b590584307e8 None None] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:46:16.016719 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1177321c-2ebc-4543-b911-807afbd0e18b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:46:16.017616 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1177321c-2ebc-4543-b911-807afbd0e18b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:46:16.017901 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1177321c-2ebc-4543-b911-807afbd0e18b admin admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:46:16.018177 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1177321c-2ebc-4543-b911-807afbd0e18b admin admin] PATH_INFO: `/v3/projects/f6c0a787b5c44452860dd16503933a35` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:46:16.019458 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1177321c-2ebc-4543-b911-807afbd0e18b admin admin] RBAC: Authorizing `identity:delete_project(project_id=f6c0a787b5c44452860dd16503933a35)` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:46:16.026637 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1177321c-2ebc-4543-b911-807afbd0e18b admin admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:46:16.087172 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-1177321c-2ebc-4543-b911-807afbd0e18b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f6c0a787b5c44452860dd16503933a35', 'request_id': 'req-1177321c-2ebc-4543-b911-807afbd0e18b'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:46:16.094785 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-1177321c-2ebc-4543-b911-807afbd0e18b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f6c0a787b5c44452860dd16503933a35', 'request_id': 'req-1177321c-2ebc-4543-b911-807afbd0e18b'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:46:16.101335 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-1177321c-2ebc-4543-b911-807afbd0e18b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f6c0a787b5c44452860dd16503933a35', 'request_id': 'req-1177321c-2ebc-4543-b911-807afbd0e18b'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:46:16.113121 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-1177321c-2ebc-4543-b911-807afbd0e18b admin admin] The token cache is being invalidate because project f6c0a787b5c44452860dd16503933a35 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80312) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 01 16:46:16.113248 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-1177321c-2ebc-4543-b911-807afbd0e18b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:46:16.113919 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-1177321c-2ebc-4543-b911-807afbd0e18b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:46:16.115648 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1233/2447] 10.222.0.57 () {58 vars in 1232 bytes} [Fri Jul 1 16:46:15 2022] DELETE /identity/v3/projects/f6c0a787b5c44452860dd16503933a35 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 16:46:16.119632 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3054034c-e1c3-4125-a527-aad809aa86aa service neutron] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:46:16.129500 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3054034c-e1c3-4125-a527-aad809aa86aa service neutron] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:46:16.186990 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f2bddc-21d9-4477-b264-461e1717f21f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:46:16.188047 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f2bddc-21d9-4477-b264-461e1717f21f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:46:16.188432 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f2bddc-21d9-4477-b264-461e1717f21f admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:46:16.188701 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f2bddc-21d9-4477-b264-461e1717f21f admin admin] PATH_INFO: `/v3/users/676a2c5e213748e5853836c7d289af69` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:46:16.190072 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f2bddc-21d9-4477-b264-461e1717f21f admin admin] RBAC: Authorizing `identity:delete_user(user_id=676a2c5e213748e5853836c7d289af69)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:46:16.192102 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f2bddc-21d9-4477-b264-461e1717f21f admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:46:16.206945 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:46:16.242964 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-b5f2bddc-21d9-4477-b264-461e1717f21f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '676a2c5e213748e5853836c7d289af69'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:46:16.243580 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-b5f2bddc-21d9-4477-b264-461e1717f21f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '676a2c5e213748e5853836c7d289af69'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:46:16.244179 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-b5f2bddc-21d9-4477-b264-461e1717f21f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '676a2c5e213748e5853836c7d289af69'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:46:16.251900 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-b5f2bddc-21d9-4477-b264-461e1717f21f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '676a2c5e213748e5853836c7d289af69'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:46:16.265077 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-b5f2bddc-21d9-4477-b264-461e1717f21f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '676a2c5e213748e5853836c7d289af69'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:46:16.279322 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1215/2448] 10.222.0.57 () {58 vars in 1223 bytes} [Fri Jul 1 16:46:16 2022] DELETE /identity/v3/users/676a2c5e213748e5853836c7d289af69 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 16:46:16.560988 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1177321c-2ebc-4543-b911-807afbd0e18b admin admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:46:16.562849 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-1177321c-2ebc-4543-b911-807afbd0e18b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:46:16.584077 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1177321c-2ebc-4543-b911-807afbd0e18b admin admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:46:16.649624 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab395a62-63a1-4292-bfd5-24f90bc559b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:46:16.650517 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab395a62-63a1-4292-bfd5-24f90bc559b2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:46:16.650802 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab395a62-63a1-4292-bfd5-24f90bc559b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:46:16.651077 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab395a62-63a1-4292-bfd5-24f90bc559b2 admin admin] PATH_INFO: `/v3/projects/0931d09302d94f90b0a2ad7d10329112` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:46:16.652258 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab395a62-63a1-4292-bfd5-24f90bc559b2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0931d09302d94f90b0a2ad7d10329112)` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:46:16.654173 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab395a62-63a1-4292-bfd5-24f90bc559b2 admin admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:46:16.701156 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-ab395a62-63a1-4292-bfd5-24f90bc559b2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0931d09302d94f90b0a2ad7d10329112', 'request_id': 'req-ab395a62-63a1-4292-bfd5-24f90bc559b2'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:46:16.711619 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-ab395a62-63a1-4292-bfd5-24f90bc559b2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0931d09302d94f90b0a2ad7d10329112', 'request_id': 'req-ab395a62-63a1-4292-bfd5-24f90bc559b2'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:46:16.717443 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-ab395a62-63a1-4292-bfd5-24f90bc559b2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0931d09302d94f90b0a2ad7d10329112', 'request_id': 'req-ab395a62-63a1-4292-bfd5-24f90bc559b2'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:46:16.726975 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:46:16.731498 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-ab395a62-63a1-4292-bfd5-24f90bc559b2 admin admin] The token cache is being invalidate because project 0931d09302d94f90b0a2ad7d10329112 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80312) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 01 16:46:16.732004 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-ab395a62-63a1-4292-bfd5-24f90bc559b2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:46:16.732949 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-ab395a62-63a1-4292-bfd5-24f90bc559b2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:46:16.735086 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1234/2449] 10.222.0.57 () {58 vars in 1232 bytes} [Fri Jul 1 16:46:16 2022] DELETE /identity/v3/projects/0931d09302d94f90b0a2ad7d10329112 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 16:46:16.740886 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6182ea9d-87f6-4be0-a959-22a451da82c7 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:46:16.741373 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6182ea9d-87f6-4be0-a959-22a451da82c7 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:46:16.741843 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6182ea9d-87f6-4be0-a959-22a451da82c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:46:16.760724 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-6182ea9d-87f6-4be0-a959-22a451da82c7 None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:46:16.761954 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-6182ea9d-87f6-4be0-a959-22a451da82c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:46:16.830439 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1216/2450] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 1 16:46:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 01 16:46:16.840507 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb86229-22cb-472a-a1e8-8e9a0b87da36 None None] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:46:16.841037 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb86229-22cb-472a-a1e8-8e9a0b87da36 None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:46:16.841503 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb86229-22cb-472a-a1e8-8e9a0b87da36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:46:16.863060 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.auth.core [None req-9fb86229-22cb-472a-a1e8-8e9a0b87da36 None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80312) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:46:16.864281 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-9fb86229-22cb-472a-a1e8-8e9a0b87da36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:46:16.874318 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1235/2451] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 1 16:46:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 16:46:16.878422 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6182ea9d-87f6-4be0-a959-22a451da82c7 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:46:16.878829 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8c4efc-483e-49c7-9ae2-3f3416db6608 None None] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:46:16.879119 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8c4efc-483e-49c7-9ae2-3f3416db6608 None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:46:16.879400 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8c4efc-483e-49c7-9ae2-3f3416db6608 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:46:16.891350 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6182ea9d-87f6-4be0-a959-22a451da82c7 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:46:16.897197 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53c7eaa-c94b-42c7-9ecc-7f799cbf6370 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:46:16.897953 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53c7eaa-c94b-42c7-9ecc-7f799cbf6370 admin admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:46:16.898314 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53c7eaa-c94b-42c7-9ecc-7f799cbf6370 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:46:16.898690 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53c7eaa-c94b-42c7-9ecc-7f799cbf6370 admin admin] PATH_INFO: `/v3/domains` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:46:16.899497 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f53c7eaa-c94b-42c7-9ecc-7f799cbf6370 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:46:16.899599 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.auth.core [None req-df8c4efc-483e-49c7-9ae2-3f3416db6608 None None] MFA Rules not processed for user `c4de1b1eeaed4fc5ae7200b2c7a90957`. Rule list: `[]` (Enabled: `True`). {{(pid=80312) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:46:16.899942 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f53c7eaa-c94b-42c7-9ecc-7f799cbf6370 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:46:16.900799 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-df8c4efc-483e-49c7-9ae2-3f3416db6608 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:46:16.901614 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f53c7eaa-c94b-42c7-9ecc-7f799cbf6370 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:46:16.914564 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1217/2452] 10.222.0.57 () {58 vars in 1152 bytes} [Fri Jul 1 16:46:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:46:16.918878 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53c7eaa-c94b-42c7-9ecc-7f799cbf6370 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:46:16.932325 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53c7eaa-c94b-42c7-9ecc-7f799cbf6370 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:46:16.937213 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa95520d-6903-4e0b-8cec-95789609f006 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:46:16.937708 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa95520d-6903-4e0b-8cec-95789609f006 admin admin] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:46:16.938000 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa95520d-6903-4e0b-8cec-95789609f006 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:46:16.938278 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa95520d-6903-4e0b-8cec-95789609f006 admin admin] PATH_INFO: `/v3/projects` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:46:16.939013 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa95520d-6903-4e0b-8cec-95789609f006 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:46:16.940922 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa95520d-6903-4e0b-8cec-95789609f006 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:46:16.966532 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1236/2453] 10.222.0.57 () {58 vars in 1009 bytes} [Fri Jul 1 16:46:16 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 16:46:16.979887 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1218/2454] 10.222.0.57 () {60 vars in 1152 bytes} [Fri Jul 1 16:46:16 2022] POST /identity/v3/projects => generated 371 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 01 16:46:16.980109 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8c4efc-483e-49c7-9ae2-3f3416db6608 None None] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:46:16.980109 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-df8c4efc-483e-49c7-9ae2-3f3416db6608 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:46:16.981373 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa95520d-6903-4e0b-8cec-95789609f006 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:46:17.045738 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=89,keystone:DELETE=9,keystone:INSERT=2 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:46:17.051853 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa95520d-6903-4e0b-8cec-95789609f006 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:46:17.055620 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8c4efc-483e-49c7-9ae2-3f3416db6608 None None] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:46:17.058461 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5fb3144-0929-421c-a992-c6910bfa1a9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:46:17.059327 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fb3144-0929-421c-a992-c6910bfa1a9d admin admin] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:46:17.059698 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fb3144-0929-421c-a992-c6910bfa1a9d admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:46:17.060144 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fb3144-0929-421c-a992-c6910bfa1a9d admin admin] PATH_INFO: `/v3/users` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:46:17.060928 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5fb3144-0929-421c-a992-c6910bfa1a9d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:46:17.062905 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5fb3144-0929-421c-a992-c6910bfa1a9d admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:46:17.085184 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1219/2455] 10.222.0.57 () {60 vars in 1143 bytes} [Fri Jul 1 16:46:16 2022] POST /identity/v3/users => generated 286 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 01 16:46:17.089631 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5fb3144-0929-421c-a992-c6910bfa1a9d admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:46:17.099879 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5fb3144-0929-421c-a992-c6910bfa1a9d admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:46:17.105008 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90250e6-1299-4620-8f06-b1241627139b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:46:17.105680 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90250e6-1299-4620-8f06-b1241627139b admin admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:46:17.105968 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90250e6-1299-4620-8f06-b1241627139b admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:46:17.106250 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90250e6-1299-4620-8f06-b1241627139b admin admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:46:17.107017 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e90250e6-1299-4620-8f06-b1241627139b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:46:17.108635 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e90250e6-1299-4620-8f06-b1241627139b admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:46:17.119237 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1220/2456] 10.222.0.57 () {58 vars in 1121 bytes} [Fri Jul 1 16:46:17 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 01 16:46:17.123192 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90250e6-1299-4620-8f06-b1241627139b admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:46:17.124060 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ade708-bf9d-455c-bf81-9c9a2c017a1a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '859fe410c2d2450fb803c6244839f27c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0788e6725a84e57a4174284c1845755', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:46:17.124817 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ade708-bf9d-455c-bf81-9c9a2c017a1a service swift] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:46:17.125227 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ade708-bf9d-455c-bf81-9c9a2c017a1a service swift] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:46:17.125588 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ade708-bf9d-455c-bf81-9c9a2c017a1a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:46:17.133416 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90250e6-1299-4620-8f06-b1241627139b admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:46:17.139594 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10ec0bd-d80a-420d-93fe-d25330058cf4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:46:17.140241 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10ec0bd-d80a-420d-93fe-d25330058cf4 admin admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:46:17.140521 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10ec0bd-d80a-420d-93fe-d25330058cf4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:46:17.140824 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10ec0bd-d80a-420d-93fe-d25330058cf4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:46:17.141548 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e10ec0bd-d80a-420d-93fe-d25330058cf4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:46:17.143506 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e10ec0bd-d80a-420d-93fe-d25330058cf4 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:46:17.155141 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1221/2457] 10.222.0.57 () {58 vars in 1121 bytes} [Fri Jul 1 16:46:17 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 16:46:17.159189 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10ec0bd-d80a-420d-93fe-d25330058cf4 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:46:17.170870 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10ec0bd-d80a-420d-93fe-d25330058cf4 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:46:17.172000 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=100 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:46:17.178446 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe34d8d-9391-4325-8ae4-4793a3027f32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:46:17.179998 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe34d8d-9391-4325-8ae4-4793a3027f32 admin admin] REQUEST_METHOD: `PUT` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:46:17.180635 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe34d8d-9391-4325-8ae4-4793a3027f32 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:46:17.181021 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe34d8d-9391-4325-8ae4-4793a3027f32 admin admin] PATH_INFO: `/v3/projects/af13e6517ccb47aa9dd9ec4c747be4be/users/643df5f09ced4c64a83208c1c67c8560/roles/983be40bdab04c478eaa90c6b6a46b14` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:46:17.188604 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ade708-bf9d-455c-bf81-9c9a2c017a1a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:46:17.190265 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ade708-bf9d-455c-bf81-9c9a2c017a1a service swift] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:46:17.200937 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1237/2458] 10.222.0.57 () {60 vars in 1384 bytes} [Fri Jul 1 16:46:16 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 223 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 01 16:46:17.202082 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe34d8d-9391-4325-8ae4-4793a3027f32 admin admin] RBAC: Authorizing `identity:create_grant(project_id=af13e6517ccb47aa9dd9ec4c747be4be, user_id=643df5f09ced4c64a83208c1c67c8560, role_id=983be40bdab04c478eaa90c6b6a46b14)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:46:17.205278 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe34d8d-9391-4325-8ae4-4793a3027f32 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:46:17.221784 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1222/2459] 10.222.0.57 () {60 vars in 1482 bytes} [Fri Jul 1 16:46:17 2022] PUT /identity/v3/projects/af13e6517ccb47aa9dd9ec4c747be4be/users/643df5f09ced4c64a83208c1c67c8560/roles/983be40bdab04c478eaa90c6b6a46b14 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 01 16:46:17.227686 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f69a37e-2f3f-4a95-b97a-5361ee2b65fa None None] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:46:17.227891 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f69a37e-2f3f-4a95-b97a-5361ee2b65fa None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:46:17.228259 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f69a37e-2f3f-4a95-b97a-5361ee2b65fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:46:17.252037 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.auth.core [None req-7f69a37e-2f3f-4a95-b97a-5361ee2b65fa None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80312) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:46:17.253650 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-7f69a37e-2f3f-4a95-b97a-5361ee2b65fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:46:17.354226 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1238/2460] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 1 16:46:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 16:46:17.366130 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe34d8d-9391-4325-8ae4-4793a3027f32 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:46:17.368244 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-9fe34d8d-9391-4325-8ae4-4793a3027f32 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:46:17.441138 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe34d8d-9391-4325-8ae4-4793a3027f32 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:46:17.445282 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372c72bd-c632-4079-b928-a411a3e9f8ca service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ed200ce29684c5f80354fa9a8638fe2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0788e6725a84e57a4174284c1845755', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:46:17.445933 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372c72bd-c632-4079-b928-a411a3e9f8ca service neutron] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:46:17.446215 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372c72bd-c632-4079-b928-a411a3e9f8ca service neutron] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:46:17.446526 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372c72bd-c632-4079-b928-a411a3e9f8ca service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:46:17.460779 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-372c72bd-c632-4079-b928-a411a3e9f8ca service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:46:17.463739 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-372c72bd-c632-4079-b928-a411a3e9f8ca service neutron] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:46:17.477134 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1223/2461] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 1 16:46:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 01 16:46:18.163836 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f69a37e-2f3f-4a95-b97a-5361ee2b65fa None None] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:46:18.166044 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-7f69a37e-2f3f-4a95-b97a-5361ee2b65fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:46:18.308615 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d346faf-98f2-4f9f-9fe5-ddaea0a6fa7b None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:46:18.309735 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d346faf-98f2-4f9f-9fe5-ddaea0a6fa7b None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:46:18.310201 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d346faf-98f2-4f9f-9fe5-ddaea0a6fa7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:46:18.316781 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f69a37e-2f3f-4a95-b97a-5361ee2b65fa None None] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:46:18.338295 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af60fe9-d17e-4dfd-8059-7a4def6eb5e6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a50f745b31941b7a7265c0fc852f48e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0788e6725a84e57a4174284c1845755', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:46:18.339279 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af60fe9-d17e-4dfd-8059-7a4def6eb5e6 service nova] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:46:18.339755 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af60fe9-d17e-4dfd-8059-7a4def6eb5e6 service nova] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:46:18.340206 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af60fe9-d17e-4dfd-8059-7a4def6eb5e6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:46:18.343162 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-9af60fe9-d17e-4dfd-8059-7a4def6eb5e6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:46:18.365966 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-6d346faf-98f2-4f9f-9fe5-ddaea0a6fa7b None None] MFA Rules not processed for user `643df5f09ced4c64a83208c1c67c8560`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:46:18.367680 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-6d346faf-98f2-4f9f-9fe5-ddaea0a6fa7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:46:18.368419 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af60fe9-d17e-4dfd-8059-7a4def6eb5e6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:46:18.370554 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af60fe9-d17e-4dfd-8059-7a4def6eb5e6 service nova] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:46:18.387833 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1239/2462] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 1 16:46:18 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 224 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 16:46:18.499311 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1224/2463] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 1 16:46:18 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 16:46:18.504437 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af60fe9-d17e-4dfd-8059-7a4def6eb5e6 service nova] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:46:18.523661 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af60fe9-d17e-4dfd-8059-7a4def6eb5e6 service nova] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:46:18.533710 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d780345b-33f1-434d-8dca-0d1e2f276cf1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:46:18.534703 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d780345b-33f1-434d-8dca-0d1e2f276cf1 admin admin] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:46:18.535215 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d780345b-33f1-434d-8dca-0d1e2f276cf1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:46:18.535711 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d780345b-33f1-434d-8dca-0d1e2f276cf1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:46:18.538017 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d780345b-33f1-434d-8dca-0d1e2f276cf1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:46:18.540130 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d780345b-33f1-434d-8dca-0d1e2f276cf1 admin admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:46:18.578195 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1240/2464] 10.222.0.57 () {60 vars in 1152 bytes} [Fri Jul 1 16:46:18 2022] POST /identity/v3/projects => generated 373 bytes in 74 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 16:46:18.584834 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d346faf-98f2-4f9f-9fe5-ddaea0a6fa7b None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:46:18.606290 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d346faf-98f2-4f9f-9fe5-ddaea0a6fa7b None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:46:18.656435 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:46:18.695112 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdba5b8f-0ef7-40fb-8385-6691442a608f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:46:18.695968 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdba5b8f-0ef7-40fb-8385-6691442a608f admin admin] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:46:18.696361 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdba5b8f-0ef7-40fb-8385-6691442a608f admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:46:18.696744 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdba5b8f-0ef7-40fb-8385-6691442a608f admin admin] PATH_INFO: `/v3/users` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:46:18.697522 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdba5b8f-0ef7-40fb-8385-6691442a608f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:46:18.699618 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdba5b8f-0ef7-40fb-8385-6691442a608f admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:46:18.725818 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1225/2465] 10.222.0.57 () {60 vars in 1143 bytes} [Fri Jul 1 16:46:18 2022] POST /identity/v3/users => generated 293 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 16:46:18.727045 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d780345b-33f1-434d-8dca-0d1e2f276cf1 admin admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:46:18.738540 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d780345b-33f1-434d-8dca-0d1e2f276cf1 admin admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:46:18.744477 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97693671-a67e-4f0d-a5a0-42e308151cdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:46:18.745227 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97693671-a67e-4f0d-a5a0-42e308151cdb admin admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:46:18.745524 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97693671-a67e-4f0d-a5a0-42e308151cdb admin admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:46:18.745806 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97693671-a67e-4f0d-a5a0-42e308151cdb admin admin] PATH_INFO: `/v3/roles` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:46:18.746587 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97693671-a67e-4f0d-a5a0-42e308151cdb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:46:18.748288 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97693671-a67e-4f0d-a5a0-42e308151cdb admin admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:46:18.767259 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1241/2466] 10.222.0.57 () {58 vars in 1121 bytes} [Fri Jul 1 16:46:18 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 16:46:18.771800 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdba5b8f-0ef7-40fb-8385-6691442a608f admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:46:18.784486 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdba5b8f-0ef7-40fb-8385-6691442a608f admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:46:18.790405 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4190740e-a650-4f19-9c45-a3e09a70d672 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:46:18.791453 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4190740e-a650-4f19-9c45-a3e09a70d672 admin admin] REQUEST_METHOD: `PUT` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:46:18.791736 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4190740e-a650-4f19-9c45-a3e09a70d672 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:46:18.792016 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4190740e-a650-4f19-9c45-a3e09a70d672 admin admin] PATH_INFO: `/v3/projects/691ef7680cd249ef8b82b4a643b1f199/users/006d1975151d472689b86f157bd9e803/roles/84ee739146934c1ab32c501ff8093644` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:46:18.811605 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4190740e-a650-4f19-9c45-a3e09a70d672 admin admin] RBAC: Authorizing `identity:create_grant(project_id=691ef7680cd249ef8b82b4a643b1f199, user_id=006d1975151d472689b86f157bd9e803, role_id=84ee739146934c1ab32c501ff8093644)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:46:18.814499 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4190740e-a650-4f19-9c45-a3e09a70d672 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:46:18.841790 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97693671-a67e-4f0d-a5a0-42e308151cdb admin admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:46:18.842217 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1226/2467] 10.222.0.57 () {60 vars in 1482 bytes} [Fri Jul 1 16:46:18 2022] PUT /identity/v3/projects/691ef7680cd249ef8b82b4a643b1f199/users/006d1975151d472689b86f157bd9e803/roles/84ee739146934c1ab32c501ff8093644 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 16:46:18.854619 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97693671-a67e-4f0d-a5a0-42e308151cdb admin admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:46:18.900432 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:46:18.935119 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9a7f72-d00a-40a1-b4ae-03c30131cc3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:46:18.935887 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9a7f72-d00a-40a1-b4ae-03c30131cc3a admin admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:46:18.936272 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9a7f72-d00a-40a1-b4ae-03c30131cc3a admin admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:46:18.936653 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9a7f72-d00a-40a1-b4ae-03c30131cc3a admin admin] PATH_INFO: `/v3/roles` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:46:18.937502 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc9a7f72-d00a-40a1-b4ae-03c30131cc3a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:46:18.939238 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc9a7f72-d00a-40a1-b4ae-03c30131cc3a admin admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:46:18.955437 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1242/2468] 10.222.0.57 () {58 vars in 1121 bytes} [Fri Jul 1 16:46:18 2022] GET /identity/v3/roles => generated 1420 bytes in 114 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 16:46:18.960207 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4190740e-a650-4f19-9c45-a3e09a70d672 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:46:18.976411 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4190740e-a650-4f19-9c45-a3e09a70d672 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:46:18.982162 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8cf296b-f312-4e2d-90e5-18d48da9102c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:46:18.983424 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cf296b-f312-4e2d-90e5-18d48da9102c admin admin] REQUEST_METHOD: `PUT` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:46:18.984230 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cf296b-f312-4e2d-90e5-18d48da9102c admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:46:18.984423 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cf296b-f312-4e2d-90e5-18d48da9102c admin admin] PATH_INFO: `/v3/projects/691ef7680cd249ef8b82b4a643b1f199/users/006d1975151d472689b86f157bd9e803/roles/84ee739146934c1ab32c501ff8093644` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:46:18.988414 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8cf296b-f312-4e2d-90e5-18d48da9102c admin admin] RBAC: Authorizing `identity:create_grant(project_id=691ef7680cd249ef8b82b4a643b1f199, user_id=006d1975151d472689b86f157bd9e803, role_id=84ee739146934c1ab32c501ff8093644)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:46:18.992107 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8cf296b-f312-4e2d-90e5-18d48da9102c admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:46:19.003400 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1227/2469] 10.222.0.57 () {60 vars in 1482 bytes} [Fri Jul 1 16:46:18 2022] PUT /identity/v3/projects/691ef7680cd249ef8b82b4a643b1f199/users/006d1975151d472689b86f157bd9e803/roles/84ee739146934c1ab32c501ff8093644 => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 16:46:20.089938 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c47200-73db-40de-b657-78ce582347da None None] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:46:20.090482 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c47200-73db-40de-b657-78ce582347da None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:46:20.090591 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c47200-73db-40de-b657-78ce582347da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:46:20.131748 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.auth.core [None req-97c47200-73db-40de-b657-78ce582347da None None] MFA Rules not processed for user `006d1975151d472689b86f157bd9e803`. Rule list: `[]` (Enabled: `True`). {{(pid=80312) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:46:20.133017 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-97c47200-73db-40de-b657-78ce582347da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:46:20.227689 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1243/2470] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 1 16:46:20 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 16:46:20.240611 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8cf296b-f312-4e2d-90e5-18d48da9102c admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:46:20.244205 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-f8cf296b-f312-4e2d-90e5-18d48da9102c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:46:20.319356 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8cf296b-f312-4e2d-90e5-18d48da9102c admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:46:20.326483 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d20588-cb2a-43ea-9c4e-1dc5ce3484af service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbddf971a2354bbdbb9da1d63f3abf88', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0788e6725a84e57a4174284c1845755', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:46:20.326483 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d20588-cb2a-43ea-9c4e-1dc5ce3484af service cinder] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:46:20.326483 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d20588-cb2a-43ea-9c4e-1dc5ce3484af service cinder] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:46:20.326483 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d20588-cb2a-43ea-9c4e-1dc5ce3484af service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:46:20.388869 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d20588-cb2a-43ea-9c4e-1dc5ce3484af service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:46:20.390334 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d20588-cb2a-43ea-9c4e-1dc5ce3484af service cinder] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:46:20.401240 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1228/2471] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 1 16:46:20 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 16:46:21.882466 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c47200-73db-40de-b657-78ce582347da None None] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:46:21.884347 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-97c47200-73db-40de-b657-78ce582347da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:46:21.898252 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c47200-73db-40de-b657-78ce582347da None None] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:46:21.904849 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ba838c-8462-4ad5-8071-3be5d5aead67 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbddf971a2354bbdbb9da1d63f3abf88', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0788e6725a84e57a4174284c1845755', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:46:21.905447 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ba838c-8462-4ad5-8071-3be5d5aead67 service cinder] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:46:21.905596 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ba838c-8462-4ad5-8071-3be5d5aead67 service cinder] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:46:21.905958 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ba838c-8462-4ad5-8071-3be5d5aead67 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:46:21.917361 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ba838c-8462-4ad5-8071-3be5d5aead67 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:46:21.918851 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ba838c-8462-4ad5-8071-3be5d5aead67 service cinder] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:46:21.932570 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1244/2472] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 1 16:46:21 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 16:46:30.397101 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=78,keystone:INSERT=5 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:46:31.926242 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=53 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:46:32.119519 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d20588-cb2a-43ea-9c4e-1dc5ce3484af service cinder] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:46:32.174127 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d20588-cb2a-43ea-9c4e-1dc5ce3484af service cinder] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:46:32.179368 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03457747-9b2b-46f3-abb2-bb4d345324e1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ed200ce29684c5f80354fa9a8638fe2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0788e6725a84e57a4174284c1845755', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:46:32.180024 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03457747-9b2b-46f3-abb2-bb4d345324e1 service neutron] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:46:32.180314 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03457747-9b2b-46f3-abb2-bb4d345324e1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:46:32.180635 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03457747-9b2b-46f3-abb2-bb4d345324e1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:46:32.182641 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-03457747-9b2b-46f3-abb2-bb4d345324e1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:46:32.193115 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03457747-9b2b-46f3-abb2-bb4d345324e1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:46:32.194567 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03457747-9b2b-46f3-abb2-bb4d345324e1 service neutron] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:46:32.202768 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1229/2473] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 1 16:46:32 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 16:46:34.572053 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec41116-228b-4aeb-873e-855d3e684ff0 None None] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:46:34.572585 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec41116-228b-4aeb-873e-855d3e684ff0 None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:46:34.572684 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec41116-228b-4aeb-873e-855d3e684ff0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:46:34.591557 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.auth.core [None req-fec41116-228b-4aeb-873e-855d3e684ff0 None None] MFA Rules not processed for user `c4de1b1eeaed4fc5ae7200b2c7a90957`. Rule list: `[]` (Enabled: `True`). {{(pid=80312) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:46:34.592779 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-fec41116-228b-4aeb-873e-855d3e684ff0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:46:34.659154 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1245/2474] 10.222.0.57 () {58 vars in 1009 bytes} [Fri Jul 1 16:46:34 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 16:46:34.674876 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03457747-9b2b-46f3-abb2-bb4d345324e1 service neutron] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:46:34.756447 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03457747-9b2b-46f3-abb2-bb4d345324e1 service neutron] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:46:34.762911 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830ae757-a528-42f6-9d8c-9b67ee8c9c47 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '859fe410c2d2450fb803c6244839f27c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0788e6725a84e57a4174284c1845755', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:46:34.763576 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830ae757-a528-42f6-9d8c-9b67ee8c9c47 service swift] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:46:34.763866 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830ae757-a528-42f6-9d8c-9b67ee8c9c47 service swift] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:46:34.764152 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830ae757-a528-42f6-9d8c-9b67ee8c9c47 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:46:34.773880 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-830ae757-a528-42f6-9d8c-9b67ee8c9c47 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:46:34.775515 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-830ae757-a528-42f6-9d8c-9b67ee8c9c47 service swift] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:46:34.784462 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1230/2475] 10.222.0.57 () {60 vars in 1384 bytes} [Fri Jul 1 16:46:34 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 01 16:46:35.437021 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909f7ea5-559b-43f6-bda9-75795af435b4 None None] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:46:35.437447 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909f7ea5-559b-43f6-bda9-75795af435b4 None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:46:35.437851 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909f7ea5-559b-43f6-bda9-75795af435b4 None None] PATH_INFO: `/` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:46:35.440346 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1246/2476] 10.222.0.57 () {54 vars in 921 bytes} [Fri Jul 1 16:46:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 16:46:35.449369 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0de3de-70e0-41c7-a27b-13b0683ea078 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:46:35.449868 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0de3de-70e0-41c7-a27b-13b0683ea078 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:46:35.450764 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0de3de-70e0-41c7-a27b-13b0683ea078 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:46:35.496470 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-da0de3de-70e0-41c7-a27b-13b0683ea078 None None] MFA Rules not processed for user `4ed200ce29684c5f80354fa9a8638fe2`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:46:35.498597 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-da0de3de-70e0-41c7-a27b-13b0683ea078 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:46:35.523144 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1231/2477] 10.222.0.57 () {58 vars in 1018 bytes} [Fri Jul 1 16:46:35 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 16:46:35.634380 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909f7ea5-559b-43f6-bda9-75795af435b4 None None] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:46:35.670802 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909f7ea5-559b-43f6-bda9-75795af435b4 None None] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:46:35.700749 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3443a315-c528-45af-99f7-0842472ebceb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ed200ce29684c5f80354fa9a8638fe2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0788e6725a84e57a4174284c1845755', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:46:35.701456 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3443a315-c528-45af-99f7-0842472ebceb service neutron] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:46:35.706791 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3443a315-c528-45af-99f7-0842472ebceb service neutron] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:46:35.708799 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3443a315-c528-45af-99f7-0842472ebceb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:46:35.730765 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3443a315-c528-45af-99f7-0842472ebceb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:46:35.735332 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3443a315-c528-45af-99f7-0842472ebceb service neutron] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:46:35.755062 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1247/2478] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 1 16:46:35 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 01 16:46:35.946397 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0649e3-34b7-4816-8be8-a2193197949f None None] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:46:35.947344 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0649e3-34b7-4816-8be8-a2193197949f None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:46:35.947798 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0649e3-34b7-4816-8be8-a2193197949f None None] PATH_INFO: `/` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:46:35.949773 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1232/2479] 10.222.0.57 () {54 vars in 921 bytes} [Fri Jul 1 16:46:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 16:46:35.962078 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd7dd39-1382-4a05-8178-eb34e344ae48 None None] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:46:35.962472 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd7dd39-1382-4a05-8178-eb34e344ae48 None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:46:35.962924 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd7dd39-1382-4a05-8178-eb34e344ae48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:46:36.061167 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.auth.core [None req-9fd7dd39-1382-4a05-8178-eb34e344ae48 None None] MFA Rules not processed for user `7a50f745b31941b7a7265c0fc852f48e`. Rule list: `[]` (Enabled: `True`). {{(pid=80312) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:46:36.062572 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-9fd7dd39-1382-4a05-8178-eb34e344ae48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:46:36.211342 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1248/2480] 10.222.0.57 () {58 vars in 1018 bytes} [Fri Jul 1 16:46:35 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 251 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 16:46:36.224648 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0649e3-34b7-4816-8be8-a2193197949f None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:46:36.232666 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-8b0649e3-34b7-4816-8be8-a2193197949f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:46:36.259941 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd7dd39-1382-4a05-8178-eb34e344ae48 None None] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:46:36.262573 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0649e3-34b7-4816-8be8-a2193197949f None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:46:36.281320 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605906c5-3f03-45e4-ab44-cda2c3883e50 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ed200ce29684c5f80354fa9a8638fe2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0788e6725a84e57a4174284c1845755', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:46:36.282422 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605906c5-3f03-45e4-ab44-cda2c3883e50 service neutron] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:46:36.290859 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605906c5-3f03-45e4-ab44-cda2c3883e50 service neutron] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:46:36.291489 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605906c5-3f03-45e4-ab44-cda2c3883e50 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:46:36.308675 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605906c5-3f03-45e4-ab44-cda2c3883e50 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:46:36.310114 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605906c5-3f03-45e4-ab44-cda2c3883e50 service neutron] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:46:36.364488 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd7dd39-1382-4a05-8178-eb34e344ae48 None None] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:46:36.365277 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1233/2481] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 1 16:46:36 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 16:46:36.369822 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d67ba93-a495-4789-9822-c571ffea7eb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:46:36.370836 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d67ba93-a495-4789-9822-c571ffea7eb6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:46:36.371194 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d67ba93-a495-4789-9822-c571ffea7eb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:46:36.371549 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d67ba93-a495-4789-9822-c571ffea7eb6 admin admin] PATH_INFO: `/v3/users/643df5f09ced4c64a83208c1c67c8560` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:46:36.372891 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d67ba93-a495-4789-9822-c571ffea7eb6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=643df5f09ced4c64a83208c1c67c8560)` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:46:36.374953 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d67ba93-a495-4789-9822-c571ffea7eb6 admin admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:46:36.392783 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:46:36.486191 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-2d67ba93-a495-4789-9822-c571ffea7eb6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '643df5f09ced4c64a83208c1c67c8560'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:46:36.500670 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-2d67ba93-a495-4789-9822-c571ffea7eb6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '643df5f09ced4c64a83208c1c67c8560'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:46:36.510193 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-2d67ba93-a495-4789-9822-c571ffea7eb6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '643df5f09ced4c64a83208c1c67c8560'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:46:36.511212 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-2d67ba93-a495-4789-9822-c571ffea7eb6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '643df5f09ced4c64a83208c1c67c8560'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:46:36.511908 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-2d67ba93-a495-4789-9822-c571ffea7eb6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '643df5f09ced4c64a83208c1c67c8560'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:46:36.542071 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1249/2482] 10.222.0.57 () {58 vars in 1223 bytes} [Fri Jul 1 16:46:36 2022] DELETE /identity/v3/users/643df5f09ced4c64a83208c1c67c8560 => generated 0 bytes in 280 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 16:46:36.814660 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605906c5-3f03-45e4-ab44-cda2c3883e50 service neutron] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:46:36.842172 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605906c5-3f03-45e4-ab44-cda2c3883e50 service neutron] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:46:36.971545 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41262b1a-f1f3-4335-85d6-241f9875ebe0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:46:36.972567 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41262b1a-f1f3-4335-85d6-241f9875ebe0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:46:36.972855 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41262b1a-f1f3-4335-85d6-241f9875ebe0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:46:36.973901 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41262b1a-f1f3-4335-85d6-241f9875ebe0 admin admin] PATH_INFO: `/v3/projects/af13e6517ccb47aa9dd9ec4c747be4be` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:46:36.975737 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41262b1a-f1f3-4335-85d6-241f9875ebe0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=af13e6517ccb47aa9dd9ec4c747be4be)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:46:36.986105 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41262b1a-f1f3-4335-85d6-241f9875ebe0 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:46:37.028615 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:46:37.085840 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-41262b1a-f1f3-4335-85d6-241f9875ebe0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'af13e6517ccb47aa9dd9ec4c747be4be', 'request_id': 'req-41262b1a-f1f3-4335-85d6-241f9875ebe0'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:46:37.091674 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-41262b1a-f1f3-4335-85d6-241f9875ebe0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'af13e6517ccb47aa9dd9ec4c747be4be', 'request_id': 'req-41262b1a-f1f3-4335-85d6-241f9875ebe0'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:46:37.099215 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-41262b1a-f1f3-4335-85d6-241f9875ebe0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'af13e6517ccb47aa9dd9ec4c747be4be', 'request_id': 'req-41262b1a-f1f3-4335-85d6-241f9875ebe0'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:46:37.116130 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-41262b1a-f1f3-4335-85d6-241f9875ebe0 admin admin] The token cache is being invalidate because project af13e6517ccb47aa9dd9ec4c747be4be was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80311) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 01 16:46:37.116711 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-41262b1a-f1f3-4335-85d6-241f9875ebe0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:46:37.117798 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-41262b1a-f1f3-4335-85d6-241f9875ebe0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:46:37.122934 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1234/2483] 10.222.0.57 () {58 vars in 1232 bytes} [Fri Jul 1 16:46:36 2022] DELETE /identity/v3/projects/af13e6517ccb47aa9dd9ec4c747be4be => generated 0 bytes in 306 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 16:46:37.124047 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d67ba93-a495-4789-9822-c571ffea7eb6 admin admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:46:37.125544 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-2d67ba93-a495-4789-9822-c571ffea7eb6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:46:37.142247 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d67ba93-a495-4789-9822-c571ffea7eb6 admin admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:46:37.201922 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa62a0b-d6bb-49ac-9359-b88efab5819f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:46:37.203168 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa62a0b-d6bb-49ac-9359-b88efab5819f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:46:37.203611 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa62a0b-d6bb-49ac-9359-b88efab5819f admin admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:46:37.204066 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa62a0b-d6bb-49ac-9359-b88efab5819f admin admin] PATH_INFO: `/v3/users/006d1975151d472689b86f157bd9e803` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:46:37.205562 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa62a0b-d6bb-49ac-9359-b88efab5819f admin admin] RBAC: Authorizing `identity:delete_user(user_id=006d1975151d472689b86f157bd9e803)` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:46:37.208463 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa62a0b-d6bb-49ac-9359-b88efab5819f admin admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:46:37.264291 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-0fa62a0b-d6bb-49ac-9359-b88efab5819f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '006d1975151d472689b86f157bd9e803'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:46:37.283672 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-0fa62a0b-d6bb-49ac-9359-b88efab5819f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '006d1975151d472689b86f157bd9e803'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:46:37.291675 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-0fa62a0b-d6bb-49ac-9359-b88efab5819f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '006d1975151d472689b86f157bd9e803'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:46:37.292516 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-0fa62a0b-d6bb-49ac-9359-b88efab5819f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '006d1975151d472689b86f157bd9e803'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:46:37.293207 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-0fa62a0b-d6bb-49ac-9359-b88efab5819f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '006d1975151d472689b86f157bd9e803'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:46:37.331850 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1250/2484] 10.222.0.57 () {58 vars in 1223 bytes} [Fri Jul 1 16:46:37 2022] DELETE /identity/v3/users/006d1975151d472689b86f157bd9e803 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 16:46:37.568961 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41262b1a-f1f3-4335-85d6-241f9875ebe0 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:46:37.570454 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-41262b1a-f1f3-4335-85d6-241f9875ebe0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:46:37.603993 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41262b1a-f1f3-4335-85d6-241f9875ebe0 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:46:37.750987 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db33c11-0b3c-4285-9f68-fa0fae7c1952 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:46:37.751991 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db33c11-0b3c-4285-9f68-fa0fae7c1952 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:46:37.752291 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db33c11-0b3c-4285-9f68-fa0fae7c1952 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:46:37.752586 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db33c11-0b3c-4285-9f68-fa0fae7c1952 admin admin] PATH_INFO: `/v3/projects/691ef7680cd249ef8b82b4a643b1f199` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:46:37.755804 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db33c11-0b3c-4285-9f68-fa0fae7c1952 admin admin] RBAC: Authorizing `identity:delete_project(project_id=691ef7680cd249ef8b82b4a643b1f199)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:46:37.757761 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db33c11-0b3c-4285-9f68-fa0fae7c1952 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:46:37.848854 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-8db33c11-0b3c-4285-9f68-fa0fae7c1952 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '691ef7680cd249ef8b82b4a643b1f199', 'request_id': 'req-8db33c11-0b3c-4285-9f68-fa0fae7c1952'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:46:37.862266 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-8db33c11-0b3c-4285-9f68-fa0fae7c1952 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '691ef7680cd249ef8b82b4a643b1f199', 'request_id': 'req-8db33c11-0b3c-4285-9f68-fa0fae7c1952'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:46:37.874216 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-8db33c11-0b3c-4285-9f68-fa0fae7c1952 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '691ef7680cd249ef8b82b4a643b1f199', 'request_id': 'req-8db33c11-0b3c-4285-9f68-fa0fae7c1952'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:46:37.901764 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-8db33c11-0b3c-4285-9f68-fa0fae7c1952 admin admin] The token cache is being invalidate because project 691ef7680cd249ef8b82b4a643b1f199 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80311) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 01 16:46:37.902072 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-8db33c11-0b3c-4285-9f68-fa0fae7c1952 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:46:37.902839 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-8db33c11-0b3c-4285-9f68-fa0fae7c1952 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:46:37.904778 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1235/2485] 10.222.0.57 () {58 vars in 1232 bytes} [Fri Jul 1 16:46:37 2022] DELETE /identity/v3/projects/691ef7680cd249ef8b82b4a643b1f199 => generated 0 bytes in 336 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 16:46:37.909560 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8c5022-17ac-4fe7-823c-e894f91a6293 None None] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:46:37.909858 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8c5022-17ac-4fe7-823c-e894f91a6293 None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:46:37.910171 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8c5022-17ac-4fe7-823c-e894f91a6293 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:46:37.948256 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.auth.core [None req-ab8c5022-17ac-4fe7-823c-e894f91a6293 None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80312) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:46:37.949531 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-ab8c5022-17ac-4fe7-823c-e894f91a6293 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:46:37.954658 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=79,keystone:DELETE=19,keystone:INSERT=2 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:46:38.020808 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1251/2486] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 1 16:46:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 16:46:38.030385 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42eb2911-3d17-4f97-8f01-f183fa080335 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:46:38.030977 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42eb2911-3d17-4f97-8f01-f183fa080335 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:46:38.031255 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42eb2911-3d17-4f97-8f01-f183fa080335 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:46:38.058685 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-42eb2911-3d17-4f97-8f01-f183fa080335 None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:46:38.059939 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-42eb2911-3d17-4f97-8f01-f183fa080335 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:46:38.071000 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1236/2487] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 1 16:46:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 16:46:38.075251 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8c5022-17ac-4fe7-823c-e894f91a6293 None None] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:46:38.086938 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8c5022-17ac-4fe7-823c-e894f91a6293 None None] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:46:38.091917 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22079c05-2feb-46e7-a5e0-881ee05babe9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:46:38.092688 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22079c05-2feb-46e7-a5e0-881ee05babe9 admin admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:46:38.093078 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22079c05-2feb-46e7-a5e0-881ee05babe9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:46:38.093429 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22079c05-2feb-46e7-a5e0-881ee05babe9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:46:38.094248 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22079c05-2feb-46e7-a5e0-881ee05babe9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80312) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:46:38.094758 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22079c05-2feb-46e7-a5e0-881ee05babe9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:46:38.096509 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22079c05-2feb-46e7-a5e0-881ee05babe9 admin admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:46:38.109644 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1252/2488] 10.222.0.57 () {58 vars in 1152 bytes} [Fri Jul 1 16:46:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:46:38.114075 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42eb2911-3d17-4f97-8f01-f183fa080335 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:46:38.123863 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42eb2911-3d17-4f97-8f01-f183fa080335 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:46:38.134269 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e4315b-4f26-4222-b0fc-e1b70be53434 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:46:38.134991 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e4315b-4f26-4222-b0fc-e1b70be53434 admin admin] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:46:38.135274 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e4315b-4f26-4222-b0fc-e1b70be53434 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:46:38.135550 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e4315b-4f26-4222-b0fc-e1b70be53434 admin admin] PATH_INFO: `/v3/projects` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:46:38.136260 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e4315b-4f26-4222-b0fc-e1b70be53434 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:46:38.138376 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e4315b-4f26-4222-b0fc-e1b70be53434 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:46:38.166436 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:46:38.181203 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1237/2489] 10.222.0.57 () {60 vars in 1152 bytes} [Fri Jul 1 16:46:38 2022] POST /identity/v3/projects => generated 385 bytes in 68 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 16:46:38.185384 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22079c05-2feb-46e7-a5e0-881ee05babe9 admin admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:46:38.195903 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22079c05-2feb-46e7-a5e0-881ee05babe9 admin admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:46:38.268362 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531ad9cd-6efa-42c0-a7a5-b4d14ce1acb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:46:38.269125 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531ad9cd-6efa-42c0-a7a5-b4d14ce1acb0 admin admin] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:46:38.269478 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531ad9cd-6efa-42c0-a7a5-b4d14ce1acb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:46:38.270272 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531ad9cd-6efa-42c0-a7a5-b4d14ce1acb0 admin admin] PATH_INFO: `/v3/users` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:46:38.271362 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-531ad9cd-6efa-42c0-a7a5-b4d14ce1acb0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:46:38.273323 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-531ad9cd-6efa-42c0-a7a5-b4d14ce1acb0 admin admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:46:38.299143 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1253/2490] 10.222.0.57 () {60 vars in 1143 bytes} [Fri Jul 1 16:46:38 2022] POST /identity/v3/users => generated 293 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 16:46:38.303671 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e4315b-4f26-4222-b0fc-e1b70be53434 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:46:38.314736 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e4315b-4f26-4222-b0fc-e1b70be53434 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:46:38.320344 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13bb7195-03d2-4e51-82c2-e26607f09bd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:46:38.321018 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bb7195-03d2-4e51-82c2-e26607f09bd4 admin admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:46:38.321305 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bb7195-03d2-4e51-82c2-e26607f09bd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:46:38.321588 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bb7195-03d2-4e51-82c2-e26607f09bd4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:46:38.322325 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13bb7195-03d2-4e51-82c2-e26607f09bd4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:46:38.324301 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13bb7195-03d2-4e51-82c2-e26607f09bd4 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:46:38.336247 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1238/2491] 10.222.0.57 () {58 vars in 1121 bytes} [Fri Jul 1 16:46:38 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 01 16:46:38.340424 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531ad9cd-6efa-42c0-a7a5-b4d14ce1acb0 admin admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:46:38.353163 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531ad9cd-6efa-42c0-a7a5-b4d14ce1acb0 admin admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:46:38.358946 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8208f846-60ff-45c2-8223-67e4cf589d4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:46:38.359968 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8208f846-60ff-45c2-8223-67e4cf589d4c admin admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:46:38.360408 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8208f846-60ff-45c2-8223-67e4cf589d4c admin admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:46:38.360966 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8208f846-60ff-45c2-8223-67e4cf589d4c admin admin] PATH_INFO: `/v3/roles` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:46:38.362071 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8208f846-60ff-45c2-8223-67e4cf589d4c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:46:38.364281 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8208f846-60ff-45c2-8223-67e4cf589d4c admin admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:46:38.376007 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1254/2492] 10.222.0.57 () {58 vars in 1121 bytes} [Fri Jul 1 16:46:38 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 16:46:38.380377 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13bb7195-03d2-4e51-82c2-e26607f09bd4 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:46:38.390450 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13bb7195-03d2-4e51-82c2-e26607f09bd4 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:46:38.395802 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3880401-53c5-443f-974d-d066731321f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:46:38.396719 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3880401-53c5-443f-974d-d066731321f4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:46:38.397092 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3880401-53c5-443f-974d-d066731321f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:46:38.397431 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3880401-53c5-443f-974d-d066731321f4 admin admin] PATH_INFO: `/v3/projects/9a94fa4f0fdc47b7bd70e372f4290f7a/users/000e90d03d4e497a857f91fe3a3289cb/roles/983be40bdab04c478eaa90c6b6a46b14` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:46:38.415668 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3880401-53c5-443f-974d-d066731321f4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9a94fa4f0fdc47b7bd70e372f4290f7a, user_id=000e90d03d4e497a857f91fe3a3289cb, role_id=983be40bdab04c478eaa90c6b6a46b14)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:46:38.418413 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3880401-53c5-443f-974d-d066731321f4 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:46:38.436330 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1239/2493] 10.222.0.57 () {60 vars in 1482 bytes} [Fri Jul 1 16:46:38 2022] PUT /identity/v3/projects/9a94fa4f0fdc47b7bd70e372f4290f7a/users/000e90d03d4e497a857f91fe3a3289cb/roles/983be40bdab04c478eaa90c6b6a46b14 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 16:46:38.448628 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632f35a5-72d0-486f-ad94-1befb8e2d1c7 None None] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:46:38.449057 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632f35a5-72d0-486f-ad94-1befb8e2d1c7 None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:46:38.449340 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632f35a5-72d0-486f-ad94-1befb8e2d1c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:46:38.476497 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.auth.core [None req-632f35a5-72d0-486f-ad94-1befb8e2d1c7 None None] MFA Rules not processed for user `000e90d03d4e497a857f91fe3a3289cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80312) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:46:38.478073 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-632f35a5-72d0-486f-ad94-1befb8e2d1c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:46:38.556003 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1255/2494] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 1 16:46:38 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 16:46:38.567125 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3880401-53c5-443f-974d-d066731321f4 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:46:38.569111 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-b3880401-53c5-443f-974d-d066731321f4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:46:38.631726 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3880401-53c5-443f-974d-d066731321f4 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:46:38.636008 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53f07f0-be70-4b8b-8ee0-e55291460853 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbddf971a2354bbdbb9da1d63f3abf88', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0788e6725a84e57a4174284c1845755', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:46:38.636672 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53f07f0-be70-4b8b-8ee0-e55291460853 service cinder] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:46:38.636960 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53f07f0-be70-4b8b-8ee0-e55291460853 service cinder] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:46:38.637423 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53f07f0-be70-4b8b-8ee0-e55291460853 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:46:38.648396 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b53f07f0-be70-4b8b-8ee0-e55291460853 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:46:38.649946 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b53f07f0-be70-4b8b-8ee0-e55291460853 service cinder] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:46:38.661556 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1240/2495] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 1 16:46:38 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 16:46:38.680517 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632f35a5-72d0-486f-ad94-1befb8e2d1c7 None None] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:46:38.685944 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:46:38.754374 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632f35a5-72d0-486f-ad94-1befb8e2d1c7 None None] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:46:38.759096 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e2fa7d-59ec-4fea-9acc-174f1f7e2c5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:46:38.759972 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e2fa7d-59ec-4fea-9acc-174f1f7e2c5c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:46:38.760258 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e2fa7d-59ec-4fea-9acc-174f1f7e2c5c admin admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:46:38.760539 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e2fa7d-59ec-4fea-9acc-174f1f7e2c5c admin admin] PATH_INFO: `/v3/users/000e90d03d4e497a857f91fe3a3289cb` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:46:38.761750 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e2fa7d-59ec-4fea-9acc-174f1f7e2c5c admin admin] RBAC: Authorizing `identity:delete_user(user_id=000e90d03d4e497a857f91fe3a3289cb)` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:46:38.763723 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e2fa7d-59ec-4fea-9acc-174f1f7e2c5c admin admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:46:38.840786 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-38e2fa7d-59ec-4fea-9acc-174f1f7e2c5c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '000e90d03d4e497a857f91fe3a3289cb'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:46:38.883102 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-38e2fa7d-59ec-4fea-9acc-174f1f7e2c5c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '000e90d03d4e497a857f91fe3a3289cb'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:46:38.899389 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-38e2fa7d-59ec-4fea-9acc-174f1f7e2c5c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '000e90d03d4e497a857f91fe3a3289cb'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:46:38.900232 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-38e2fa7d-59ec-4fea-9acc-174f1f7e2c5c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '000e90d03d4e497a857f91fe3a3289cb'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:46:38.900900 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-38e2fa7d-59ec-4fea-9acc-174f1f7e2c5c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '000e90d03d4e497a857f91fe3a3289cb'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:46:38.916056 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1256/2496] 10.222.0.57 () {58 vars in 1223 bytes} [Fri Jul 1 16:46:38 2022] DELETE /identity/v3/users/000e90d03d4e497a857f91fe3a3289cb => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 16:46:38.921096 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73efa283-3a3d-4a8e-9798-69d11546c554 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:46:38.921324 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73efa283-3a3d-4a8e-9798-69d11546c554 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:46:38.921606 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73efa283-3a3d-4a8e-9798-69d11546c554 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:46:38.971589 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-73efa283-3a3d-4a8e-9798-69d11546c554 None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:46:38.972944 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-73efa283-3a3d-4a8e-9798-69d11546c554 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:46:39.059659 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1241/2497] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 1 16:46:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 01 16:46:39.070458 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e2fa7d-59ec-4fea-9acc-174f1f7e2c5c admin admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:46:39.071939 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-38e2fa7d-59ec-4fea-9acc-174f1f7e2c5c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:46:39.132390 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e2fa7d-59ec-4fea-9acc-174f1f7e2c5c admin admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:46:39.136771 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a3fa84-bf25-4e78-9faa-ab5aca279103 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ed200ce29684c5f80354fa9a8638fe2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0788e6725a84e57a4174284c1845755', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:46:39.137496 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a3fa84-bf25-4e78-9faa-ab5aca279103 service neutron] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:46:39.138332 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a3fa84-bf25-4e78-9faa-ab5aca279103 service neutron] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:46:39.138651 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a3fa84-bf25-4e78-9faa-ab5aca279103 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:46:39.152185 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a3fa84-bf25-4e78-9faa-ab5aca279103 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:46:39.153746 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a3fa84-bf25-4e78-9faa-ab5aca279103 service neutron] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:46:39.171958 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1257/2498] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 1 16:46:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 16:46:39.561744 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73efa283-3a3d-4a8e-9798-69d11546c554 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:46:39.563204 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-73efa283-3a3d-4a8e-9798-69d11546c554 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:46:39.584452 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73efa283-3a3d-4a8e-9798-69d11546c554 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:46:39.589270 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e48035-e181-4e13-aec7-222a039fba90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:46:39.590148 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e48035-e181-4e13-aec7-222a039fba90 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:46:39.590474 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e48035-e181-4e13-aec7-222a039fba90 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:46:39.590738 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e48035-e181-4e13-aec7-222a039fba90 admin admin] PATH_INFO: `/v3/projects/9a94fa4f0fdc47b7bd70e372f4290f7a` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:46:39.591951 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e48035-e181-4e13-aec7-222a039fba90 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9a94fa4f0fdc47b7bd70e372f4290f7a)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:46:39.593935 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e48035-e181-4e13-aec7-222a039fba90 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:46:39.660568 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-c6e48035-e181-4e13-aec7-222a039fba90 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9a94fa4f0fdc47b7bd70e372f4290f7a', 'request_id': 'req-c6e48035-e181-4e13-aec7-222a039fba90'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:46:39.673536 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-c6e48035-e181-4e13-aec7-222a039fba90 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9a94fa4f0fdc47b7bd70e372f4290f7a', 'request_id': 'req-c6e48035-e181-4e13-aec7-222a039fba90'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:46:39.675078 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=93,keystone:INSERT=1,keystone:DELETE=6 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:46:39.683539 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-c6e48035-e181-4e13-aec7-222a039fba90 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9a94fa4f0fdc47b7bd70e372f4290f7a', 'request_id': 'req-c6e48035-e181-4e13-aec7-222a039fba90'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:46:39.699835 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-c6e48035-e181-4e13-aec7-222a039fba90 admin admin] The token cache is being invalidate because project 9a94fa4f0fdc47b7bd70e372f4290f7a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80311) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 01 16:46:39.700119 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-c6e48035-e181-4e13-aec7-222a039fba90 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:46:39.700764 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-c6e48035-e181-4e13-aec7-222a039fba90 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:46:39.705700 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1242/2499] 10.222.0.57 () {58 vars in 1232 bytes} [Fri Jul 1 16:46:39 2022] DELETE /identity/v3/projects/9a94fa4f0fdc47b7bd70e372f4290f7a => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 16:46:39.707615 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fb9717-e4b5-4f7d-903a-d8aeb7bf2aab None None] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:46:39.708350 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fb9717-e4b5-4f7d-903a-d8aeb7bf2aab None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:46:39.708500 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fb9717-e4b5-4f7d-903a-d8aeb7bf2aab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:46:39.736286 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.auth.core [None req-05fb9717-e4b5-4f7d-903a-d8aeb7bf2aab None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80312) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:46:39.737445 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-05fb9717-e4b5-4f7d-903a-d8aeb7bf2aab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:46:39.759615 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:46:39.806893 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1258/2500] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 1 16:46:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 16:46:39.820858 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf6d172-b3b4-4c93-89d5-972b8b3db3ca None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:46:39.821111 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf6d172-b3b4-4c93-89d5-972b8b3db3ca None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:46:39.821394 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf6d172-b3b4-4c93-89d5-972b8b3db3ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:46:39.841852 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-abf6d172-b3b4-4c93-89d5-972b8b3db3ca None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:46:39.843273 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-abf6d172-b3b4-4c93-89d5-972b8b3db3ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:46:39.858400 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1243/2501] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 1 16:46:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 16:46:39.861471 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05fb9717-e4b5-4f7d-903a-d8aeb7bf2aab None None] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:46:39.872831 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05fb9717-e4b5-4f7d-903a-d8aeb7bf2aab None None] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:46:39.877483 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ebda87-d45b-4d0c-b5be-527ffbb351e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:46:39.878150 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ebda87-d45b-4d0c-b5be-527ffbb351e8 admin admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:46:39.878473 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ebda87-d45b-4d0c-b5be-527ffbb351e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:46:39.878756 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ebda87-d45b-4d0c-b5be-527ffbb351e8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:46:39.879780 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ebda87-d45b-4d0c-b5be-527ffbb351e8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80312) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:46:39.880094 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ebda87-d45b-4d0c-b5be-527ffbb351e8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:46:39.881719 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ebda87-d45b-4d0c-b5be-527ffbb351e8 admin admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:46:39.893691 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1259/2502] 10.222.0.57 () {58 vars in 1152 bytes} [Fri Jul 1 16:46:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:46:39.898967 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf6d172-b3b4-4c93-89d5-972b8b3db3ca None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:46:39.913482 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf6d172-b3b4-4c93-89d5-972b8b3db3ca None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:46:39.918634 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f844945a-8c39-4678-afca-d1f1eb0b0554 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:46:39.919352 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f844945a-8c39-4678-afca-d1f1eb0b0554 admin admin] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:46:39.919638 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f844945a-8c39-4678-afca-d1f1eb0b0554 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:46:39.919914 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f844945a-8c39-4678-afca-d1f1eb0b0554 admin admin] PATH_INFO: `/v3/projects` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:46:39.920615 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f844945a-8c39-4678-afca-d1f1eb0b0554 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:46:39.922587 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f844945a-8c39-4678-afca-d1f1eb0b0554 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:46:39.953749 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1244/2503] 10.222.0.57 () {60 vars in 1152 bytes} [Fri Jul 1 16:46:39 2022] POST /identity/v3/projects => generated 381 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 16:46:39.958476 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ebda87-d45b-4d0c-b5be-527ffbb351e8 admin admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:46:39.969368 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ebda87-d45b-4d0c-b5be-527ffbb351e8 admin admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:46:40.059168 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f13b30f-4d7e-4b19-b653-266544382260 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:46:40.059908 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f13b30f-4d7e-4b19-b653-266544382260 admin admin] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:46:40.060198 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f13b30f-4d7e-4b19-b653-266544382260 admin admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:46:40.060521 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f13b30f-4d7e-4b19-b653-266544382260 admin admin] PATH_INFO: `/v3/users` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:46:40.061301 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f13b30f-4d7e-4b19-b653-266544382260 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:46:40.063361 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f13b30f-4d7e-4b19-b653-266544382260 admin admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:46:40.091642 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1260/2504] 10.222.0.57 () {60 vars in 1143 bytes} [Fri Jul 1 16:46:39 2022] POST /identity/v3/users => generated 291 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 16:46:40.092822 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f844945a-8c39-4678-afca-d1f1eb0b0554 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:46:40.105155 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f844945a-8c39-4678-afca-d1f1eb0b0554 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:46:40.109899 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f98398-37a7-4a9d-ab7b-e6f01a73d353 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:46:40.110698 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f98398-37a7-4a9d-ab7b-e6f01a73d353 admin admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:46:40.110983 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f98398-37a7-4a9d-ab7b-e6f01a73d353 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:46:40.111263 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f98398-37a7-4a9d-ab7b-e6f01a73d353 admin admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:46:40.112037 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f98398-37a7-4a9d-ab7b-e6f01a73d353 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:46:40.113698 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f98398-37a7-4a9d-ab7b-e6f01a73d353 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:46:40.125464 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1245/2505] 10.222.0.57 () {58 vars in 1121 bytes} [Fri Jul 1 16:46:40 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 01 16:46:40.130646 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f13b30f-4d7e-4b19-b653-266544382260 admin admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:46:40.140940 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f13b30f-4d7e-4b19-b653-266544382260 admin admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:46:40.146821 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be8ddeb-e1aa-4d49-8673-995ba71fe1e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:46:40.147498 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be8ddeb-e1aa-4d49-8673-995ba71fe1e8 admin admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:46:40.147779 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be8ddeb-e1aa-4d49-8673-995ba71fe1e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:46:40.148053 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be8ddeb-e1aa-4d49-8673-995ba71fe1e8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:46:40.148784 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be8ddeb-e1aa-4d49-8673-995ba71fe1e8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:46:40.150491 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be8ddeb-e1aa-4d49-8673-995ba71fe1e8 admin admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:46:40.164740 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1261/2506] 10.222.0.57 () {58 vars in 1121 bytes} [Fri Jul 1 16:46:40 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 16:46:40.166040 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f98398-37a7-4a9d-ab7b-e6f01a73d353 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:46:40.178899 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f98398-37a7-4a9d-ab7b-e6f01a73d353 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:46:40.193676 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965e8861-9336-4f53-b4fd-836c57953ed0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:46:40.194638 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965e8861-9336-4f53-b4fd-836c57953ed0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:46:40.195033 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965e8861-9336-4f53-b4fd-836c57953ed0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:46:40.195192 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965e8861-9336-4f53-b4fd-836c57953ed0 admin admin] PATH_INFO: `/v3/projects/66bb091979114297a724ba5549e3bcdd/users/eac706db9daa49e19ee3ca1959fabc97/roles/983be40bdab04c478eaa90c6b6a46b14` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:46:40.224330 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-965e8861-9336-4f53-b4fd-836c57953ed0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=66bb091979114297a724ba5549e3bcdd, user_id=eac706db9daa49e19ee3ca1959fabc97, role_id=983be40bdab04c478eaa90c6b6a46b14)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:46:40.227074 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-965e8861-9336-4f53-b4fd-836c57953ed0 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:46:40.255831 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1246/2507] 10.222.0.57 () {60 vars in 1482 bytes} [Fri Jul 1 16:46:40 2022] PUT /identity/v3/projects/66bb091979114297a724ba5549e3bcdd/users/eac706db9daa49e19ee3ca1959fabc97/roles/983be40bdab04c478eaa90c6b6a46b14 => generated 0 bytes in 91 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 16:46:40.266593 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4352012-2f59-42f6-b673-db3882acee26 None None] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:46:40.266830 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4352012-2f59-42f6-b673-db3882acee26 None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:46:40.267119 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4352012-2f59-42f6-b673-db3882acee26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:46:40.304069 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.auth.core [None req-b4352012-2f59-42f6-b673-db3882acee26 None None] MFA Rules not processed for user `eac706db9daa49e19ee3ca1959fabc97`. Rule list: `[]` (Enabled: `True`). {{(pid=80312) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:46:40.305725 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-b4352012-2f59-42f6-b673-db3882acee26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:46:40.382601 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1262/2508] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 1 16:46:40 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 16:46:40.394711 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965e8861-9336-4f53-b4fd-836c57953ed0 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:46:40.396675 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-965e8861-9336-4f53-b4fd-836c57953ed0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:46:40.472253 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965e8861-9336-4f53-b4fd-836c57953ed0 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:46:40.477220 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02705fe8-841c-4451-b46c-230802507628 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbddf971a2354bbdbb9da1d63f3abf88', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0788e6725a84e57a4174284c1845755', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:46:40.478153 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02705fe8-841c-4451-b46c-230802507628 service cinder] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:46:40.478507 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02705fe8-841c-4451-b46c-230802507628 service cinder] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:46:40.478839 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02705fe8-841c-4451-b46c-230802507628 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:46:40.490938 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02705fe8-841c-4451-b46c-230802507628 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:46:40.492807 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02705fe8-841c-4451-b46c-230802507628 service cinder] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:46:40.517147 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1247/2509] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 1 16:46:40 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 01 16:46:49.188042 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4352012-2f59-42f6-b673-db3882acee26 None None] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:46:49.213595 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:46:49.252302 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4352012-2f59-42f6-b673-db3882acee26 None None] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:46:49.257619 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbca2d3-c927-48db-977e-8934b2bbbd4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:46:49.258524 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbca2d3-c927-48db-977e-8934b2bbbd4e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:46:49.258814 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbca2d3-c927-48db-977e-8934b2bbbd4e admin admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:46:49.259094 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbca2d3-c927-48db-977e-8934b2bbbd4e admin admin] PATH_INFO: `/v3/users/eac706db9daa49e19ee3ca1959fabc97` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:46:49.260298 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfbca2d3-c927-48db-977e-8934b2bbbd4e admin admin] RBAC: Authorizing `identity:delete_user(user_id=eac706db9daa49e19ee3ca1959fabc97)` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:46:49.262248 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfbca2d3-c927-48db-977e-8934b2bbbd4e admin admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:46:49.311024 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-cfbca2d3-c927-48db-977e-8934b2bbbd4e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eac706db9daa49e19ee3ca1959fabc97'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:46:49.323611 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-cfbca2d3-c927-48db-977e-8934b2bbbd4e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eac706db9daa49e19ee3ca1959fabc97'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:46:49.330214 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-cfbca2d3-c927-48db-977e-8934b2bbbd4e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eac706db9daa49e19ee3ca1959fabc97'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:46:49.330984 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-cfbca2d3-c927-48db-977e-8934b2bbbd4e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eac706db9daa49e19ee3ca1959fabc97'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:46:49.331577 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-cfbca2d3-c927-48db-977e-8934b2bbbd4e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eac706db9daa49e19ee3ca1959fabc97'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:46:49.345677 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1263/2510] 10.222.0.57 () {58 vars in 1223 bytes} [Fri Jul 1 16:46:49 2022] DELETE /identity/v3/users/eac706db9daa49e19ee3ca1959fabc97 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 16:46:49.350612 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f614131-a30c-4b5a-97f7-89ebfc6be9a2 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:46:49.351193 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f614131-a30c-4b5a-97f7-89ebfc6be9a2 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:46:49.351193 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f614131-a30c-4b5a-97f7-89ebfc6be9a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:46:49.370390 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-4f614131-a30c-4b5a-97f7-89ebfc6be9a2 None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:46:49.371585 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-4f614131-a30c-4b5a-97f7-89ebfc6be9a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:46:49.429850 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1248/2511] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 1 16:46:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 16:46:49.439425 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbca2d3-c927-48db-977e-8934b2bbbd4e admin admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:46:49.440863 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-cfbca2d3-c927-48db-977e-8934b2bbbd4e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:46:49.498936 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbca2d3-c927-48db-977e-8934b2bbbd4e admin admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:46:49.502561 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a07ae4-df41-4947-a14b-36458be85fe0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ed200ce29684c5f80354fa9a8638fe2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0788e6725a84e57a4174284c1845755', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:46:49.503205 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a07ae4-df41-4947-a14b-36458be85fe0 service neutron] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:46:49.503490 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a07ae4-df41-4947-a14b-36458be85fe0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:46:49.504314 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a07ae4-df41-4947-a14b-36458be85fe0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:46:49.513571 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a07ae4-df41-4947-a14b-36458be85fe0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:46:49.515032 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a07ae4-df41-4947-a14b-36458be85fe0 service neutron] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:46:49.524530 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1264/2512] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 1 16:46:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 16:46:49.853597 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f614131-a30c-4b5a-97f7-89ebfc6be9a2 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:46:49.855041 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-4f614131-a30c-4b5a-97f7-89ebfc6be9a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:46:49.867990 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f614131-a30c-4b5a-97f7-89ebfc6be9a2 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:46:49.873368 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b71b15-f353-4ea6-977a-b9bf299798b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:46:49.874479 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b71b15-f353-4ea6-977a-b9bf299798b7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:46:49.874844 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b71b15-f353-4ea6-977a-b9bf299798b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:46:49.875199 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b71b15-f353-4ea6-977a-b9bf299798b7 admin admin] PATH_INFO: `/v3/projects/66bb091979114297a724ba5549e3bcdd` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:46:49.876944 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b71b15-f353-4ea6-977a-b9bf299798b7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=66bb091979114297a724ba5549e3bcdd)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:46:49.879656 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b71b15-f353-4ea6-977a-b9bf299798b7 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:46:49.909568 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=96,keystone:INSERT=3,keystone:DELETE=1 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:46:49.949206 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-01b71b15-f353-4ea6-977a-b9bf299798b7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '66bb091979114297a724ba5549e3bcdd', 'request_id': 'req-01b71b15-f353-4ea6-977a-b9bf299798b7'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:46:49.955711 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-01b71b15-f353-4ea6-977a-b9bf299798b7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '66bb091979114297a724ba5549e3bcdd', 'request_id': 'req-01b71b15-f353-4ea6-977a-b9bf299798b7'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:46:49.963196 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-01b71b15-f353-4ea6-977a-b9bf299798b7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '66bb091979114297a724ba5549e3bcdd', 'request_id': 'req-01b71b15-f353-4ea6-977a-b9bf299798b7'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:46:49.986862 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-01b71b15-f353-4ea6-977a-b9bf299798b7 admin admin] The token cache is being invalidate because project 66bb091979114297a724ba5549e3bcdd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80311) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 01 16:46:49.987415 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-01b71b15-f353-4ea6-977a-b9bf299798b7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:46:49.988394 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-01b71b15-f353-4ea6-977a-b9bf299798b7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:46:49.990791 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1249/2513] 10.222.0.57 () {58 vars in 1232 bytes} [Fri Jul 1 16:46:49 2022] DELETE /identity/v3/projects/66bb091979114297a724ba5549e3bcdd => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 16:46:49.995468 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfdfd02-82fd-4137-87f3-4138667f6856 None None] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:46:49.995760 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfdfd02-82fd-4137-87f3-4138667f6856 None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:46:49.996041 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfdfd02-82fd-4137-87f3-4138667f6856 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:46:50.017020 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.auth.core [None req-6dfdfd02-82fd-4137-87f3-4138667f6856 None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80312) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:46:50.018288 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-6dfdfd02-82fd-4137-87f3-4138667f6856 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:46:50.083791 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:46:50.117634 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1265/2514] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 1 16:46:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 16:46:50.125311 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17cfb21d-d0ab-4002-9437-2524cf7c99ac None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:46:50.125602 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17cfb21d-d0ab-4002-9437-2524cf7c99ac None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:46:50.125883 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17cfb21d-d0ab-4002-9437-2524cf7c99ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:46:50.146150 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-17cfb21d-d0ab-4002-9437-2524cf7c99ac None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:46:50.147423 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-17cfb21d-d0ab-4002-9437-2524cf7c99ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:46:50.155891 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1250/2515] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 1 16:46:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 16:46:50.160802 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dfdfd02-82fd-4137-87f3-4138667f6856 None None] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:46:50.170843 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dfdfd02-82fd-4137-87f3-4138667f6856 None None] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:46:50.175971 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343a0cfd-84ad-49e4-8db3-7578a94eae1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:46:50.176658 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343a0cfd-84ad-49e4-8db3-7578a94eae1c admin admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:46:50.176927 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343a0cfd-84ad-49e4-8db3-7578a94eae1c admin admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:46:50.177211 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343a0cfd-84ad-49e4-8db3-7578a94eae1c admin admin] PATH_INFO: `/v3/domains` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:46:50.177957 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-343a0cfd-84ad-49e4-8db3-7578a94eae1c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80312) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:46:50.178274 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-343a0cfd-84ad-49e4-8db3-7578a94eae1c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:46:50.179949 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-343a0cfd-84ad-49e4-8db3-7578a94eae1c admin admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:46:50.193692 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1266/2516] 10.222.0.57 () {58 vars in 1152 bytes} [Fri Jul 1 16:46:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:46:50.197928 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17cfb21d-d0ab-4002-9437-2524cf7c99ac None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:46:50.213166 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17cfb21d-d0ab-4002-9437-2524cf7c99ac None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:46:50.219592 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33f4409-9181-48bd-a04e-bee1afa8e4e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:46:50.220265 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33f4409-9181-48bd-a04e-bee1afa8e4e3 admin admin] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:46:50.220598 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33f4409-9181-48bd-a04e-bee1afa8e4e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:46:50.220821 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33f4409-9181-48bd-a04e-bee1afa8e4e3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:46:50.221532 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b33f4409-9181-48bd-a04e-bee1afa8e4e3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:46:50.224084 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b33f4409-9181-48bd-a04e-bee1afa8e4e3 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:46:50.252815 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1251/2517] 10.222.0.57 () {60 vars in 1152 bytes} [Fri Jul 1 16:46:50 2022] POST /identity/v3/projects => generated 373 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 16:46:50.257892 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343a0cfd-84ad-49e4-8db3-7578a94eae1c admin admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:46:50.271128 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343a0cfd-84ad-49e4-8db3-7578a94eae1c admin admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:46:50.340477 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39bcf987-96f7-4dfa-ab91-911a7a34bc51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:46:50.341163 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bcf987-96f7-4dfa-ab91-911a7a34bc51 admin admin] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:46:50.341448 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bcf987-96f7-4dfa-ab91-911a7a34bc51 admin admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:46:50.341730 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bcf987-96f7-4dfa-ab91-911a7a34bc51 admin admin] PATH_INFO: `/v3/users` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:46:50.342513 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39bcf987-96f7-4dfa-ab91-911a7a34bc51 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:46:50.344400 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39bcf987-96f7-4dfa-ab91-911a7a34bc51 admin admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:46:50.371043 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1267/2518] 10.222.0.57 () {60 vars in 1143 bytes} [Fri Jul 1 16:46:50 2022] POST /identity/v3/users => generated 287 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 16:46:50.374955 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33f4409-9181-48bd-a04e-bee1afa8e4e3 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:46:50.386139 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33f4409-9181-48bd-a04e-bee1afa8e4e3 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:46:50.390845 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90202c2e-dc13-4b7a-bb0e-07304949da44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:46:50.391509 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90202c2e-dc13-4b7a-bb0e-07304949da44 admin admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:46:50.391791 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90202c2e-dc13-4b7a-bb0e-07304949da44 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:46:50.392074 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90202c2e-dc13-4b7a-bb0e-07304949da44 admin admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:46:50.392799 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90202c2e-dc13-4b7a-bb0e-07304949da44 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:46:50.395669 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90202c2e-dc13-4b7a-bb0e-07304949da44 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:46:50.406400 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1252/2519] 10.222.0.57 () {58 vars in 1121 bytes} [Fri Jul 1 16:46:50 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 16:46:50.411641 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39bcf987-96f7-4dfa-ab91-911a7a34bc51 admin admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:46:50.421689 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39bcf987-96f7-4dfa-ab91-911a7a34bc51 admin admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:46:50.428202 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807374c7-2067-4e59-8d5b-30301f2389a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:46:50.428496 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807374c7-2067-4e59-8d5b-30301f2389a0 admin admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:46:50.428801 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807374c7-2067-4e59-8d5b-30301f2389a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:46:50.429091 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807374c7-2067-4e59-8d5b-30301f2389a0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:46:50.429855 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-807374c7-2067-4e59-8d5b-30301f2389a0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:46:50.431529 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-807374c7-2067-4e59-8d5b-30301f2389a0 admin admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:46:50.444862 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1268/2520] 10.222.0.57 () {58 vars in 1121 bytes} [Fri Jul 1 16:46:50 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 16:46:50.449256 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90202c2e-dc13-4b7a-bb0e-07304949da44 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:46:50.462400 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90202c2e-dc13-4b7a-bb0e-07304949da44 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:46:50.466993 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841e5c1b-1814-44ac-b393-89f905a5c65f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:46:50.467875 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841e5c1b-1814-44ac-b393-89f905a5c65f admin admin] REQUEST_METHOD: `PUT` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:46:50.468158 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841e5c1b-1814-44ac-b393-89f905a5c65f admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:46:50.468427 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841e5c1b-1814-44ac-b393-89f905a5c65f admin admin] PATH_INFO: `/v3/projects/8901e944300845ff94ca861be04c5624/users/a2c9568d6f73482caf1751dacf04bae4/roles/983be40bdab04c478eaa90c6b6a46b14` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:46:50.484735 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-841e5c1b-1814-44ac-b393-89f905a5c65f admin admin] RBAC: Authorizing `identity:create_grant(project_id=8901e944300845ff94ca861be04c5624, user_id=a2c9568d6f73482caf1751dacf04bae4, role_id=983be40bdab04c478eaa90c6b6a46b14)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:46:50.487612 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-841e5c1b-1814-44ac-b393-89f905a5c65f admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:46:50.510163 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1253/2521] 10.222.0.57 () {60 vars in 1482 bytes} [Fri Jul 1 16:46:50 2022] PUT /identity/v3/projects/8901e944300845ff94ca861be04c5624/users/a2c9568d6f73482caf1751dacf04bae4/roles/983be40bdab04c478eaa90c6b6a46b14 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 16:46:50.521891 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b96239-9c68-43e9-ba29-1cafdb4f485a None None] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:46:50.522311 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b96239-9c68-43e9-ba29-1cafdb4f485a None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:46:50.522762 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b96239-9c68-43e9-ba29-1cafdb4f485a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:46:50.552089 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.auth.core [None req-72b96239-9c68-43e9-ba29-1cafdb4f485a None None] MFA Rules not processed for user `a2c9568d6f73482caf1751dacf04bae4`. Rule list: `[]` (Enabled: `True`). {{(pid=80312) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:46:50.553291 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-72b96239-9c68-43e9-ba29-1cafdb4f485a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:46:50.616596 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1269/2522] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 1 16:46:50 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 16:46:50.627335 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841e5c1b-1814-44ac-b393-89f905a5c65f admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:46:50.629490 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-841e5c1b-1814-44ac-b393-89f905a5c65f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:46:50.692687 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841e5c1b-1814-44ac-b393-89f905a5c65f admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:46:50.698327 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8685259e-190c-4000-9b47-f1fadabb7315 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbddf971a2354bbdbb9da1d63f3abf88', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0788e6725a84e57a4174284c1845755', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:46:50.699501 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8685259e-190c-4000-9b47-f1fadabb7315 service cinder] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:46:50.700033 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8685259e-190c-4000-9b47-f1fadabb7315 service cinder] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:46:50.700564 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8685259e-190c-4000-9b47-f1fadabb7315 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:46:50.711239 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8685259e-190c-4000-9b47-f1fadabb7315 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:46:50.713972 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8685259e-190c-4000-9b47-f1fadabb7315 service cinder] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:46:50.727209 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1254/2523] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 1 16:46:50 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 16:46:52.184521 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3521c791-6765-45d2-915c-98367fecfb6a None None] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:46:52.185023 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3521c791-6765-45d2-915c-98367fecfb6a None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:46:52.185245 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3521c791-6765-45d2-915c-98367fecfb6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:46:52.203243 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.auth.core [None req-3521c791-6765-45d2-915c-98367fecfb6a None None] MFA Rules not processed for user `c4de1b1eeaed4fc5ae7200b2c7a90957`. Rule list: `[]` (Enabled: `True`). {{(pid=80312) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:46:52.204431 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-3521c791-6765-45d2-915c-98367fecfb6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:46:52.239123 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:46:52.263714 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1270/2524] 10.222.0.57 () {58 vars in 967 bytes} [Fri Jul 1 16:46:52 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 01 16:46:52.276890 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8685259e-190c-4000-9b47-f1fadabb7315 service cinder] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:46:52.277872 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-8685259e-190c-4000-9b47-f1fadabb7315 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:46:52.336281 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:DELETE=7,keystone:SELECT=90,keystone:INSERT=3 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:46:52.371474 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8685259e-190c-4000-9b47-f1fadabb7315 service cinder] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:46:52.378009 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f76bd4-9bdf-43c3-a1a6-81737ff3a7bc service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '859fe410c2d2450fb803c6244839f27c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0788e6725a84e57a4174284c1845755', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:46:52.388359 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f76bd4-9bdf-43c3-a1a6-81737ff3a7bc service swift] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:46:52.388715 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f76bd4-9bdf-43c3-a1a6-81737ff3a7bc service swift] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:46:52.389007 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f76bd4-9bdf-43c3-a1a6-81737ff3a7bc service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:46:52.400091 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f76bd4-9bdf-43c3-a1a6-81737ff3a7bc service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:46:52.401721 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f76bd4-9bdf-43c3-a1a6-81737ff3a7bc service swift] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:46:52.413319 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1255/2525] 10.222.0.57 () {60 vars in 1384 bytes} [Fri Jul 1 16:46:52 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 01 16:46:53.854304 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3521c791-6765-45d2-915c-98367fecfb6a None None] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:46:53.921945 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3521c791-6765-45d2-915c-98367fecfb6a None None] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:46:53.927536 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f281bad-2284-4e04-9e49-a43443a8956e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:46:53.928417 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f281bad-2284-4e04-9e49-a43443a8956e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:46:53.928778 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f281bad-2284-4e04-9e49-a43443a8956e admin admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:46:53.929002 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f281bad-2284-4e04-9e49-a43443a8956e admin admin] PATH_INFO: `/v3/users/a2c9568d6f73482caf1751dacf04bae4` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:46:53.930172 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f281bad-2284-4e04-9e49-a43443a8956e admin admin] RBAC: Authorizing `identity:delete_user(user_id=a2c9568d6f73482caf1751dacf04bae4)` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:46:53.932166 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f281bad-2284-4e04-9e49-a43443a8956e admin admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:46:53.983336 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-4f281bad-2284-4e04-9e49-a43443a8956e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a2c9568d6f73482caf1751dacf04bae4'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:46:53.996493 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-4f281bad-2284-4e04-9e49-a43443a8956e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a2c9568d6f73482caf1751dacf04bae4'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:46:54.005118 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-4f281bad-2284-4e04-9e49-a43443a8956e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a2c9568d6f73482caf1751dacf04bae4'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:46:54.005842 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-4f281bad-2284-4e04-9e49-a43443a8956e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a2c9568d6f73482caf1751dacf04bae4'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:46:54.006529 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-4f281bad-2284-4e04-9e49-a43443a8956e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a2c9568d6f73482caf1751dacf04bae4'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:46:54.021127 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1271/2526] 10.222.0.57 () {58 vars in 1223 bytes} [Fri Jul 1 16:46:53 2022] DELETE /identity/v3/users/a2c9568d6f73482caf1751dacf04bae4 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 16:46:54.025959 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4e585c-09bd-45b0-b00a-46ad5cbc2483 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:46:54.026466 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4e585c-09bd-45b0-b00a-46ad5cbc2483 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:46:54.026567 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4e585c-09bd-45b0-b00a-46ad5cbc2483 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:46:54.044380 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-7e4e585c-09bd-45b0-b00a-46ad5cbc2483 None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:46:54.045580 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-7e4e585c-09bd-45b0-b00a-46ad5cbc2483 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:46:54.105129 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1256/2527] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 1 16:46:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 16:46:54.115349 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f281bad-2284-4e04-9e49-a43443a8956e admin admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:46:54.116791 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-4f281bad-2284-4e04-9e49-a43443a8956e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:46:54.175960 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f281bad-2284-4e04-9e49-a43443a8956e admin admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:46:54.179723 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203d2f40-9695-4e28-9c01-9702e428f3af service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ed200ce29684c5f80354fa9a8638fe2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0788e6725a84e57a4174284c1845755', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:46:54.180371 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203d2f40-9695-4e28-9c01-9702e428f3af service neutron] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:46:54.180655 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203d2f40-9695-4e28-9c01-9702e428f3af service neutron] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:46:54.180929 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203d2f40-9695-4e28-9c01-9702e428f3af service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:46:54.190427 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-203d2f40-9695-4e28-9c01-9702e428f3af service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:46:54.191922 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-203d2f40-9695-4e28-9c01-9702e428f3af service neutron] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:46:54.203319 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1272/2528] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 1 16:46:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 16:46:54.513800 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4e585c-09bd-45b0-b00a-46ad5cbc2483 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:46:54.515698 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-7e4e585c-09bd-45b0-b00a-46ad5cbc2483 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:46:54.529502 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4e585c-09bd-45b0-b00a-46ad5cbc2483 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:46:54.536798 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768d3e03-a844-4798-874c-ba0a3a5a8a6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:46:54.537857 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768d3e03-a844-4798-874c-ba0a3a5a8a6b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:46:54.538214 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768d3e03-a844-4798-874c-ba0a3a5a8a6b admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:46:54.538609 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768d3e03-a844-4798-874c-ba0a3a5a8a6b admin admin] PATH_INFO: `/v3/projects/8901e944300845ff94ca861be04c5624` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:46:54.539930 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-768d3e03-a844-4798-874c-ba0a3a5a8a6b admin admin] RBAC: Authorizing `identity:delete_project(project_id=8901e944300845ff94ca861be04c5624)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:46:54.542133 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-768d3e03-a844-4798-874c-ba0a3a5a8a6b admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:46:54.593736 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-768d3e03-a844-4798-874c-ba0a3a5a8a6b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8901e944300845ff94ca861be04c5624', 'request_id': 'req-768d3e03-a844-4798-874c-ba0a3a5a8a6b'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:46:54.600372 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-768d3e03-a844-4798-874c-ba0a3a5a8a6b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8901e944300845ff94ca861be04c5624', 'request_id': 'req-768d3e03-a844-4798-874c-ba0a3a5a8a6b'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:46:54.606615 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-768d3e03-a844-4798-874c-ba0a3a5a8a6b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8901e944300845ff94ca861be04c5624', 'request_id': 'req-768d3e03-a844-4798-874c-ba0a3a5a8a6b'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:46:54.620474 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-768d3e03-a844-4798-874c-ba0a3a5a8a6b admin admin] The token cache is being invalidate because project 8901e944300845ff94ca861be04c5624 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80311) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 01 16:46:54.620713 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-768d3e03-a844-4798-874c-ba0a3a5a8a6b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:46:54.621377 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-768d3e03-a844-4798-874c-ba0a3a5a8a6b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:46:54.623123 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1257/2529] 10.222.0.57 () {58 vars in 1232 bytes} [Fri Jul 1 16:46:54 2022] DELETE /identity/v3/projects/8901e944300845ff94ca861be04c5624 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 16:46:54.628299 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365e15d4-29a6-4a16-bb12-d3ae5c575417 None None] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:46:54.629248 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365e15d4-29a6-4a16-bb12-d3ae5c575417 None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:46:54.629717 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365e15d4-29a6-4a16-bb12-d3ae5c575417 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:46:54.643670 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:46:54.650418 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.auth.core [None req-365e15d4-29a6-4a16-bb12-d3ae5c575417 None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80312) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:46:54.652440 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-365e15d4-29a6-4a16-bb12-d3ae5c575417 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:46:54.718651 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1273/2530] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 1 16:46:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 16:46:54.783529 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0008028-a31e-4986-b72f-2e3a73ac0039 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:46:54.783908 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0008028-a31e-4986-b72f-2e3a73ac0039 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:46:54.784090 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0008028-a31e-4986-b72f-2e3a73ac0039 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:46:54.809291 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-b0008028-a31e-4986-b72f-2e3a73ac0039 None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:46:54.811390 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-b0008028-a31e-4986-b72f-2e3a73ac0039 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:46:54.820153 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1258/2531] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 1 16:46:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 16:46:54.824630 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365e15d4-29a6-4a16-bb12-d3ae5c575417 None None] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:46:54.836118 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365e15d4-29a6-4a16-bb12-d3ae5c575417 None None] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:46:54.842213 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdaefa8f-6954-44fb-9e75-6498b0494255 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:46:54.843050 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdaefa8f-6954-44fb-9e75-6498b0494255 admin admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:46:54.843335 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdaefa8f-6954-44fb-9e75-6498b0494255 admin admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:46:54.843613 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdaefa8f-6954-44fb-9e75-6498b0494255 admin admin] PATH_INFO: `/v3/domains` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:46:54.846547 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdaefa8f-6954-44fb-9e75-6498b0494255 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80312) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:46:54.846547 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdaefa8f-6954-44fb-9e75-6498b0494255 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:46:54.846547 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdaefa8f-6954-44fb-9e75-6498b0494255 admin admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:46:54.862384 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1274/2532] 10.222.0.57 () {58 vars in 1152 bytes} [Fri Jul 1 16:46:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:46:54.865245 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0008028-a31e-4986-b72f-2e3a73ac0039 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:46:54.876464 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0008028-a31e-4986-b72f-2e3a73ac0039 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:46:54.881433 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ed9b12-5749-463f-a646-963fe1821801 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:46:54.882101 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ed9b12-5749-463f-a646-963fe1821801 admin admin] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:46:54.882418 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ed9b12-5749-463f-a646-963fe1821801 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:46:54.882699 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ed9b12-5749-463f-a646-963fe1821801 admin admin] PATH_INFO: `/v3/projects` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:46:54.883392 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ed9b12-5749-463f-a646-963fe1821801 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:46:54.885291 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ed9b12-5749-463f-a646-963fe1821801 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:46:54.915851 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1259/2533] 10.222.0.57 () {60 vars in 1152 bytes} [Fri Jul 1 16:46:54 2022] POST /identity/v3/projects => generated 399 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 16:46:54.916928 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdaefa8f-6954-44fb-9e75-6498b0494255 admin admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:46:54.927543 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdaefa8f-6954-44fb-9e75-6498b0494255 admin admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:46:54.984065 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086ff2d9-074f-4aae-a273-357a5046b561 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:46:54.984728 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086ff2d9-074f-4aae-a273-357a5046b561 admin admin] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:46:54.985011 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086ff2d9-074f-4aae-a273-357a5046b561 admin admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:46:54.985288 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086ff2d9-074f-4aae-a273-357a5046b561 admin admin] PATH_INFO: `/v3/users` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:46:54.985972 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-086ff2d9-074f-4aae-a273-357a5046b561 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:46:54.988685 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-086ff2d9-074f-4aae-a273-357a5046b561 admin admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:46:55.008944 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1275/2534] 10.222.0.57 () {60 vars in 1143 bytes} [Fri Jul 1 16:46:54 2022] POST /identity/v3/users => generated 300 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 01 16:46:55.013266 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ed9b12-5749-463f-a646-963fe1821801 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:46:55.024013 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ed9b12-5749-463f-a646-963fe1821801 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:46:55.028694 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170a314f-1ee1-4f39-b0b8-dbf36270707e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:46:55.030030 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170a314f-1ee1-4f39-b0b8-dbf36270707e admin admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:46:55.030320 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170a314f-1ee1-4f39-b0b8-dbf36270707e admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:46:55.030634 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170a314f-1ee1-4f39-b0b8-dbf36270707e admin admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:46:55.031366 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-170a314f-1ee1-4f39-b0b8-dbf36270707e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:46:55.032970 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-170a314f-1ee1-4f39-b0b8-dbf36270707e admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:46:55.045585 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1260/2535] 10.222.0.57 () {58 vars in 1121 bytes} [Fri Jul 1 16:46:55 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 16:46:55.049786 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086ff2d9-074f-4aae-a273-357a5046b561 admin admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:46:55.061196 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086ff2d9-074f-4aae-a273-357a5046b561 admin admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:46:55.068052 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d4bfa9-7a0e-4d5c-9ee6-b123026b0f59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:46:55.069185 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d4bfa9-7a0e-4d5c-9ee6-b123026b0f59 admin admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:46:55.069690 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d4bfa9-7a0e-4d5c-9ee6-b123026b0f59 admin admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:46:55.070177 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d4bfa9-7a0e-4d5c-9ee6-b123026b0f59 admin admin] PATH_INFO: `/v3/roles` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:46:55.071553 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d4bfa9-7a0e-4d5c-9ee6-b123026b0f59 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:46:55.073981 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d4bfa9-7a0e-4d5c-9ee6-b123026b0f59 admin admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:46:55.089769 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1276/2536] 10.222.0.57 () {58 vars in 1121 bytes} [Fri Jul 1 16:46:55 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 16:46:55.094076 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170a314f-1ee1-4f39-b0b8-dbf36270707e admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:46:55.106578 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170a314f-1ee1-4f39-b0b8-dbf36270707e admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:46:55.111427 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15cd1be0-fec0-4cbf-af60-7857ee9fb668 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:46:55.112372 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15cd1be0-fec0-4cbf-af60-7857ee9fb668 admin admin] REQUEST_METHOD: `PUT` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:46:55.112655 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15cd1be0-fec0-4cbf-af60-7857ee9fb668 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:46:55.113516 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15cd1be0-fec0-4cbf-af60-7857ee9fb668 admin admin] PATH_INFO: `/v3/projects/d47112865ac3420a8fe9c59cd74eb8b7/users/074cf644da384ada968af2284539a228/roles/983be40bdab04c478eaa90c6b6a46b14` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:46:55.131960 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15cd1be0-fec0-4cbf-af60-7857ee9fb668 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d47112865ac3420a8fe9c59cd74eb8b7, user_id=074cf644da384ada968af2284539a228, role_id=983be40bdab04c478eaa90c6b6a46b14)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:46:55.134598 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15cd1be0-fec0-4cbf-af60-7857ee9fb668 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:46:55.142711 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:46:55.148839 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1261/2537] 10.222.0.57 () {60 vars in 1482 bytes} [Fri Jul 1 16:46:55 2022] PUT /identity/v3/projects/d47112865ac3420a8fe9c59cd74eb8b7/users/074cf644da384ada968af2284539a228/roles/983be40bdab04c478eaa90c6b6a46b14 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 01 16:46:55.160707 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b382212b-d5c8-4241-b42a-a0f051d339ea None None] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:46:55.160914 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b382212b-d5c8-4241-b42a-a0f051d339ea None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:46:55.161194 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b382212b-d5c8-4241-b42a-a0f051d339ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:46:55.188516 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.auth.core [None req-b382212b-d5c8-4241-b42a-a0f051d339ea None None] MFA Rules not processed for user `074cf644da384ada968af2284539a228`. Rule list: `[]` (Enabled: `True`). {{(pid=80312) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:46:55.189908 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-b382212b-d5c8-4241-b42a-a0f051d339ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:46:55.253916 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1277/2538] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 1 16:46:55 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 16:46:55.265222 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15cd1be0-fec0-4cbf-af60-7857ee9fb668 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:46:55.266941 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-15cd1be0-fec0-4cbf-af60-7857ee9fb668 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:46:55.326717 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15cd1be0-fec0-4cbf-af60-7857ee9fb668 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:46:55.330392 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c41472-45cb-49e1-a82c-e12c48692539 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbddf971a2354bbdbb9da1d63f3abf88', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0788e6725a84e57a4174284c1845755', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:46:55.331044 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c41472-45cb-49e1-a82c-e12c48692539 service cinder] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:46:55.331335 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c41472-45cb-49e1-a82c-e12c48692539 service cinder] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:46:55.332141 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c41472-45cb-49e1-a82c-e12c48692539 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:46:55.340768 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c41472-45cb-49e1-a82c-e12c48692539 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:46:55.342189 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c41472-45cb-49e1-a82c-e12c48692539 service cinder] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:46:55.352042 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1262/2539] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 1 16:46:55 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 16:47:02.465694 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b382212b-d5c8-4241-b42a-a0f051d339ea None None] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:47:02.470013 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:47:02.545238 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b382212b-d5c8-4241-b42a-a0f051d339ea None None] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:47:02.553048 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34cd022f-7d2a-4558-a076-cfb90e699b00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:47:02.554457 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cd022f-7d2a-4558-a076-cfb90e699b00 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:47:02.554775 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cd022f-7d2a-4558-a076-cfb90e699b00 admin admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:47:02.555217 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cd022f-7d2a-4558-a076-cfb90e699b00 admin admin] PATH_INFO: `/v3/users/074cf644da384ada968af2284539a228` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:47:02.557258 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34cd022f-7d2a-4558-a076-cfb90e699b00 admin admin] RBAC: Authorizing `identity:delete_user(user_id=074cf644da384ada968af2284539a228)` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:47:02.559864 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34cd022f-7d2a-4558-a076-cfb90e699b00 admin admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:47:02.622496 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-34cd022f-7d2a-4558-a076-cfb90e699b00 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '074cf644da384ada968af2284539a228'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:47:02.638240 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-34cd022f-7d2a-4558-a076-cfb90e699b00 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '074cf644da384ada968af2284539a228'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:47:02.647787 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-34cd022f-7d2a-4558-a076-cfb90e699b00 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '074cf644da384ada968af2284539a228'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:47:02.648733 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-34cd022f-7d2a-4558-a076-cfb90e699b00 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '074cf644da384ada968af2284539a228'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:47:02.649734 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-34cd022f-7d2a-4558-a076-cfb90e699b00 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '074cf644da384ada968af2284539a228'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:47:02.664241 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1278/2540] 10.222.0.57 () {58 vars in 1223 bytes} [Fri Jul 1 16:47:02 2022] DELETE /identity/v3/users/074cf644da384ada968af2284539a228 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 16:47:02.668492 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f7f1b3-113c-4b18-8617-701f505bb2ef None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:47:02.668959 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f7f1b3-113c-4b18-8617-701f505bb2ef None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:47:02.669091 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f7f1b3-113c-4b18-8617-701f505bb2ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:47:02.696634 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-c4f7f1b3-113c-4b18-8617-701f505bb2ef None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:47:02.697959 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-c4f7f1b3-113c-4b18-8617-701f505bb2ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:47:02.762556 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1263/2541] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 1 16:47:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 16:47:02.773401 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34cd022f-7d2a-4558-a076-cfb90e699b00 admin admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:47:02.775534 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-34cd022f-7d2a-4558-a076-cfb90e699b00 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:47:02.837788 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34cd022f-7d2a-4558-a076-cfb90e699b00 admin admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:47:02.841602 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df0451c-044e-4005-98af-bc225b90c717 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ed200ce29684c5f80354fa9a8638fe2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0788e6725a84e57a4174284c1845755', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:47:02.842280 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df0451c-044e-4005-98af-bc225b90c717 service neutron] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:47:02.842754 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df0451c-044e-4005-98af-bc225b90c717 service neutron] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:47:02.843046 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df0451c-044e-4005-98af-bc225b90c717 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:47:02.854123 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df0451c-044e-4005-98af-bc225b90c717 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:47:02.855611 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df0451c-044e-4005-98af-bc225b90c717 service neutron] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:47:02.867174 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1279/2542] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 1 16:47:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 16:47:03.198319 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f7f1b3-113c-4b18-8617-701f505bb2ef None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:47:03.199880 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-c4f7f1b3-113c-4b18-8617-701f505bb2ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:47:03.214698 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f7f1b3-113c-4b18-8617-701f505bb2ef None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:47:03.219643 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a6ca19-db96-4791-8ad0-bdc87a1b58d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:47:03.220642 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a6ca19-db96-4791-8ad0-bdc87a1b58d6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:47:03.220961 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a6ca19-db96-4791-8ad0-bdc87a1b58d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:47:03.221208 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a6ca19-db96-4791-8ad0-bdc87a1b58d6 admin admin] PATH_INFO: `/v3/projects/d47112865ac3420a8fe9c59cd74eb8b7` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:47:03.223090 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a6ca19-db96-4791-8ad0-bdc87a1b58d6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d47112865ac3420a8fe9c59cd74eb8b7)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:47:03.225483 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a6ca19-db96-4791-8ad0-bdc87a1b58d6 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:47:03.277859 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-71a6ca19-db96-4791-8ad0-bdc87a1b58d6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd47112865ac3420a8fe9c59cd74eb8b7', 'request_id': 'req-71a6ca19-db96-4791-8ad0-bdc87a1b58d6'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:47:03.284023 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-71a6ca19-db96-4791-8ad0-bdc87a1b58d6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd47112865ac3420a8fe9c59cd74eb8b7', 'request_id': 'req-71a6ca19-db96-4791-8ad0-bdc87a1b58d6'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:47:03.290563 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-71a6ca19-db96-4791-8ad0-bdc87a1b58d6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd47112865ac3420a8fe9c59cd74eb8b7', 'request_id': 'req-71a6ca19-db96-4791-8ad0-bdc87a1b58d6'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:47:03.303040 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-71a6ca19-db96-4791-8ad0-bdc87a1b58d6 admin admin] The token cache is being invalidate because project d47112865ac3420a8fe9c59cd74eb8b7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80311) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 01 16:47:03.303334 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-71a6ca19-db96-4791-8ad0-bdc87a1b58d6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:47:03.303972 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-71a6ca19-db96-4791-8ad0-bdc87a1b58d6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:47:03.305761 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1264/2543] 10.222.0.57 () {58 vars in 1232 bytes} [Fri Jul 1 16:47:03 2022] DELETE /identity/v3/projects/d47112865ac3420a8fe9c59cd74eb8b7 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 16:47:07.281669 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df0451c-044e-4005-98af-bc225b90c717 service neutron] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:47:07.283083 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-1df0451c-044e-4005-98af-bc225b90c717 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:47:07.294151 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df0451c-044e-4005-98af-bc225b90c717 service neutron] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:47:07.313562 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:47:07.348088 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757721d4-e11f-4a7b-a9c8-f969fd06f186 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ed200ce29684c5f80354fa9a8638fe2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0788e6725a84e57a4174284c1845755', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:47:07.348760 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757721d4-e11f-4a7b-a9c8-f969fd06f186 service neutron] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:47:07.349052 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757721d4-e11f-4a7b-a9c8-f969fd06f186 service neutron] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:47:07.349369 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757721d4-e11f-4a7b-a9c8-f969fd06f186 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:47:07.351473 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-757721d4-e11f-4a7b-a9c8-f969fd06f186 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:47:07.414639 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-757721d4-e11f-4a7b-a9c8-f969fd06f186 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:47:07.416079 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-757721d4-e11f-4a7b-a9c8-f969fd06f186 service neutron] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:47:07.426441 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1280/2544] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 1 16:47:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 16:47:10.632114 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a6ca19-db96-4791-8ad0-bdc87a1b58d6 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:47:10.634027 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-71a6ca19-db96-4791-8ad0-bdc87a1b58d6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:47:10.649385 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a6ca19-db96-4791-8ad0-bdc87a1b58d6 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:47:10.654152 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33575985-730b-4e2a-9268-825395f0418a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:47:10.655074 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33575985-730b-4e2a-9268-825395f0418a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:47:10.655360 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33575985-730b-4e2a-9268-825395f0418a admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:47:10.655639 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33575985-730b-4e2a-9268-825395f0418a admin admin] PATH_INFO: `/v3/users/7c4da21fbfa3472cac6e766a2f4799d8` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:47:10.656829 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33575985-730b-4e2a-9268-825395f0418a admin admin] RBAC: Authorizing `identity:delete_user(user_id=7c4da21fbfa3472cac6e766a2f4799d8)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:47:10.660842 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33575985-730b-4e2a-9268-825395f0418a admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:47:10.684243 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:47:10.710635 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-33575985-730b-4e2a-9268-825395f0418a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7c4da21fbfa3472cac6e766a2f4799d8'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:47:10.711327 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-33575985-730b-4e2a-9268-825395f0418a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7c4da21fbfa3472cac6e766a2f4799d8'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:47:10.711938 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-33575985-730b-4e2a-9268-825395f0418a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7c4da21fbfa3472cac6e766a2f4799d8'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:47:10.719209 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-33575985-730b-4e2a-9268-825395f0418a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7c4da21fbfa3472cac6e766a2f4799d8'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:47:10.732894 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-33575985-730b-4e2a-9268-825395f0418a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7c4da21fbfa3472cac6e766a2f4799d8'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:47:10.745698 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1265/2545] 10.222.0.57 () {58 vars in 1223 bytes} [Fri Jul 1 16:47:10 2022] DELETE /identity/v3/users/7c4da21fbfa3472cac6e766a2f4799d8 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 16:47:10.917730 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757721d4-e11f-4a7b-a9c8-f969fd06f186 service neutron] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:47:10.928197 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757721d4-e11f-4a7b-a9c8-f969fd06f186 service neutron] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:47:10.988794 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86465c32-19b3-43e7-8f9f-a7c3d5af5bc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:47:10.989616 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86465c32-19b3-43e7-8f9f-a7c3d5af5bc2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:47:10.989899 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86465c32-19b3-43e7-8f9f-a7c3d5af5bc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:47:10.990173 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86465c32-19b3-43e7-8f9f-a7c3d5af5bc2 admin admin] PATH_INFO: `/v3/projects/5747a83d930f4075a78e9b14abd58c7f` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:47:10.991418 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86465c32-19b3-43e7-8f9f-a7c3d5af5bc2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5747a83d930f4075a78e9b14abd58c7f)` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:47:10.993344 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86465c32-19b3-43e7-8f9f-a7c3d5af5bc2 admin admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:47:11.044734 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-86465c32-19b3-43e7-8f9f-a7c3d5af5bc2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5747a83d930f4075a78e9b14abd58c7f', 'request_id': 'req-86465c32-19b3-43e7-8f9f-a7c3d5af5bc2'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:47:11.052634 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-86465c32-19b3-43e7-8f9f-a7c3d5af5bc2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5747a83d930f4075a78e9b14abd58c7f', 'request_id': 'req-86465c32-19b3-43e7-8f9f-a7c3d5af5bc2'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:47:11.058213 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-86465c32-19b3-43e7-8f9f-a7c3d5af5bc2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5747a83d930f4075a78e9b14abd58c7f', 'request_id': 'req-86465c32-19b3-43e7-8f9f-a7c3d5af5bc2'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:47:11.070141 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-86465c32-19b3-43e7-8f9f-a7c3d5af5bc2 admin admin] The token cache is being invalidate because project 5747a83d930f4075a78e9b14abd58c7f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80312) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 01 16:47:11.070454 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-86465c32-19b3-43e7-8f9f-a7c3d5af5bc2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:47:11.071085 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-86465c32-19b3-43e7-8f9f-a7c3d5af5bc2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:47:11.072618 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1281/2546] 10.222.0.57 () {58 vars in 1232 bytes} [Fri Jul 1 16:47:10 2022] DELETE /identity/v3/projects/5747a83d930f4075a78e9b14abd58c7f => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 01 16:47:11.077542 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fed58a0-4a3e-414a-b53e-bac17bf6fca7 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:47:11.077833 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fed58a0-4a3e-414a-b53e-bac17bf6fca7 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:47:11.078110 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fed58a0-4a3e-414a-b53e-bac17bf6fca7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:47:11.097384 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-6fed58a0-4a3e-414a-b53e-bac17bf6fca7 None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:47:11.098523 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-6fed58a0-4a3e-414a-b53e-bac17bf6fca7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:47:11.156205 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1266/2547] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 1 16:47:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 16:47:11.164528 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae28e17-1812-4456-af45-e4680d6ece9a None None] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:47:11.164815 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae28e17-1812-4456-af45-e4680d6ece9a None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:47:11.165092 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae28e17-1812-4456-af45-e4680d6ece9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:47:11.172271 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:47:11.184535 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.auth.core [None req-6ae28e17-1812-4456-af45-e4680d6ece9a None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80312) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:47:11.186272 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-6ae28e17-1812-4456-af45-e4680d6ece9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:47:11.194773 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1282/2548] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 1 16:47:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 16:47:11.199322 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fed58a0-4a3e-414a-b53e-bac17bf6fca7 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:47:11.210739 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fed58a0-4a3e-414a-b53e-bac17bf6fca7 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:47:11.215384 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b88ebf3-3985-4145-b57d-5a4b329600b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:47:11.216043 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b88ebf3-3985-4145-b57d-5a4b329600b6 admin admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:47:11.216330 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b88ebf3-3985-4145-b57d-5a4b329600b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:47:11.216608 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b88ebf3-3985-4145-b57d-5a4b329600b6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:47:11.217341 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b88ebf3-3985-4145-b57d-5a4b329600b6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:47:11.217650 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b88ebf3-3985-4145-b57d-5a4b329600b6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:47:11.219284 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b88ebf3-3985-4145-b57d-5a4b329600b6 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:47:11.231922 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1267/2549] 10.222.0.57 () {58 vars in 1152 bytes} [Fri Jul 1 16:47:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:47:11.236263 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae28e17-1812-4456-af45-e4680d6ece9a None None] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:47:11.245676 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae28e17-1812-4456-af45-e4680d6ece9a None None] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:47:11.250306 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2429c2-f5fd-44d9-a4ba-6fafe6b0c7ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:47:11.250989 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2429c2-f5fd-44d9-a4ba-6fafe6b0c7ae admin admin] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:47:11.251277 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2429c2-f5fd-44d9-a4ba-6fafe6b0c7ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:47:11.251553 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2429c2-f5fd-44d9-a4ba-6fafe6b0c7ae admin admin] PATH_INFO: `/v3/projects` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:47:11.252240 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb2429c2-f5fd-44d9-a4ba-6fafe6b0c7ae admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:47:11.254287 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb2429c2-f5fd-44d9-a4ba-6fafe6b0c7ae admin admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:47:11.287909 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1283/2550] 10.222.0.57 () {60 vars in 1152 bytes} [Fri Jul 1 16:47:11 2022] POST /identity/v3/projects => generated 373 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 01 16:47:11.291849 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b88ebf3-3985-4145-b57d-5a4b329600b6 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:47:11.302114 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b88ebf3-3985-4145-b57d-5a4b329600b6 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:47:11.360712 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7cdd975-2531-4bef-8beb-6a9c06890aaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:47:11.361290 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7cdd975-2531-4bef-8beb-6a9c06890aaa admin admin] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:47:11.361575 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7cdd975-2531-4bef-8beb-6a9c06890aaa admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:47:11.361852 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7cdd975-2531-4bef-8beb-6a9c06890aaa admin admin] PATH_INFO: `/v3/users` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:47:11.362689 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7cdd975-2531-4bef-8beb-6a9c06890aaa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:47:11.364618 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7cdd975-2531-4bef-8beb-6a9c06890aaa admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:47:11.383892 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1268/2551] 10.222.0.57 () {60 vars in 1143 bytes} [Fri Jul 1 16:47:11 2022] POST /identity/v3/users => generated 287 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 16:47:11.388120 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2429c2-f5fd-44d9-a4ba-6fafe6b0c7ae admin admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:47:11.399433 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2429c2-f5fd-44d9-a4ba-6fafe6b0c7ae admin admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:47:11.404219 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6f8287-436d-4b5e-b1f1-2d647c8ca601 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:47:11.404879 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6f8287-436d-4b5e-b1f1-2d647c8ca601 admin admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:47:11.405166 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6f8287-436d-4b5e-b1f1-2d647c8ca601 admin admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:47:11.405445 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6f8287-436d-4b5e-b1f1-2d647c8ca601 admin admin] PATH_INFO: `/v3/roles` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:47:11.406171 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a6f8287-436d-4b5e-b1f1-2d647c8ca601 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:47:11.407804 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a6f8287-436d-4b5e-b1f1-2d647c8ca601 admin admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:47:11.418880 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1284/2552] 10.222.0.57 () {58 vars in 1121 bytes} [Fri Jul 1 16:47:11 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 16:47:11.422915 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7cdd975-2531-4bef-8beb-6a9c06890aaa admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:47:11.433056 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7cdd975-2531-4bef-8beb-6a9c06890aaa admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:47:11.438099 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6e0275-ebab-4ab9-b7ea-ac4742137e9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:47:11.438796 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6e0275-ebab-4ab9-b7ea-ac4742137e9e admin admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:47:11.439088 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6e0275-ebab-4ab9-b7ea-ac4742137e9e admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:47:11.439374 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6e0275-ebab-4ab9-b7ea-ac4742137e9e admin admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:47:11.440180 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a6e0275-ebab-4ab9-b7ea-ac4742137e9e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:47:11.441794 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a6e0275-ebab-4ab9-b7ea-ac4742137e9e admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:47:11.451815 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1269/2553] 10.222.0.57 () {58 vars in 1121 bytes} [Fri Jul 1 16:47:11 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 16:47:11.456586 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6f8287-436d-4b5e-b1f1-2d647c8ca601 admin admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:47:11.467094 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6f8287-436d-4b5e-b1f1-2d647c8ca601 admin admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:47:11.471916 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf9f008-b1a1-417f-bc53-1f54ad454e5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:47:11.472796 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf9f008-b1a1-417f-bc53-1f54ad454e5c admin admin] REQUEST_METHOD: `PUT` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:47:11.473083 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf9f008-b1a1-417f-bc53-1f54ad454e5c admin admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:47:11.473364 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf9f008-b1a1-417f-bc53-1f54ad454e5c admin admin] PATH_INFO: `/v3/projects/db98a6214dec4c0c9cbca95c8788171f/users/a4a4e810e8b240eb99f0b5faf77ece0e/roles/983be40bdab04c478eaa90c6b6a46b14` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:47:11.487966 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf9f008-b1a1-417f-bc53-1f54ad454e5c admin admin] RBAC: Authorizing `identity:create_grant(project_id=db98a6214dec4c0c9cbca95c8788171f, user_id=a4a4e810e8b240eb99f0b5faf77ece0e, role_id=983be40bdab04c478eaa90c6b6a46b14)` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:47:11.490633 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf9f008-b1a1-417f-bc53-1f54ad454e5c admin admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:47:11.503989 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1285/2554] 10.222.0.57 () {60 vars in 1482 bytes} [Fri Jul 1 16:47:11 2022] PUT /identity/v3/projects/db98a6214dec4c0c9cbca95c8788171f/users/a4a4e810e8b240eb99f0b5faf77ece0e/roles/983be40bdab04c478eaa90c6b6a46b14 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 16:47:11.515849 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71eb4af4-1e05-4c44-9f3d-07d8c3ff263b None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:47:11.516149 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71eb4af4-1e05-4c44-9f3d-07d8c3ff263b None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:47:11.516435 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71eb4af4-1e05-4c44-9f3d-07d8c3ff263b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:47:11.523415 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:DELETE=8,keystone:SELECT=88,keystone:INSERT=4 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:47:11.548269 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-71eb4af4-1e05-4c44-9f3d-07d8c3ff263b None None] MFA Rules not processed for user `a4a4e810e8b240eb99f0b5faf77ece0e`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:47:11.550194 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-71eb4af4-1e05-4c44-9f3d-07d8c3ff263b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:47:11.615306 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1270/2555] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 1 16:47:11 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 16:47:11.626473 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf9f008-b1a1-417f-bc53-1f54ad454e5c admin admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:47:11.628549 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-daf9f008-b1a1-417f-bc53-1f54ad454e5c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:47:11.710117 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf9f008-b1a1-417f-bc53-1f54ad454e5c admin admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:47:11.716149 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696f88ad-380c-4336-8e2a-75dee12ceb67 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a50f745b31941b7a7265c0fc852f48e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0788e6725a84e57a4174284c1845755', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:47:11.716813 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696f88ad-380c-4336-8e2a-75dee12ceb67 service nova] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:47:11.717103 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696f88ad-380c-4336-8e2a-75dee12ceb67 service nova] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:47:11.717510 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696f88ad-380c-4336-8e2a-75dee12ceb67 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:47:11.726236 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-696f88ad-380c-4336-8e2a-75dee12ceb67 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:47:11.727887 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-696f88ad-380c-4336-8e2a-75dee12ceb67 service nova] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:47:11.736893 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1286/2556] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 1 16:47:11 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 16:47:11.961008 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71eb4af4-1e05-4c44-9f3d-07d8c3ff263b None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:47:12.024018 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71eb4af4-1e05-4c44-9f3d-07d8c3ff263b None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:47:12.028554 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc834fe-c1c6-4198-ab03-d83316e2b92f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:47:12.029429 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc834fe-c1c6-4198-ab03-d83316e2b92f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:47:12.029705 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc834fe-c1c6-4198-ab03-d83316e2b92f admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:47:12.029987 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc834fe-c1c6-4198-ab03-d83316e2b92f admin admin] PATH_INFO: `/v3/users/a4a4e810e8b240eb99f0b5faf77ece0e` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:47:12.031209 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc834fe-c1c6-4198-ab03-d83316e2b92f admin admin] RBAC: Authorizing `identity:delete_user(user_id=a4a4e810e8b240eb99f0b5faf77ece0e)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:47:12.033150 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc834fe-c1c6-4198-ab03-d83316e2b92f admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:47:12.078524 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-dbc834fe-c1c6-4198-ab03-d83316e2b92f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a4a4e810e8b240eb99f0b5faf77ece0e'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:47:12.079170 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-dbc834fe-c1c6-4198-ab03-d83316e2b92f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a4a4e810e8b240eb99f0b5faf77ece0e'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:47:12.079762 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-dbc834fe-c1c6-4198-ab03-d83316e2b92f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a4a4e810e8b240eb99f0b5faf77ece0e'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:47:12.087855 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-dbc834fe-c1c6-4198-ab03-d83316e2b92f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a4a4e810e8b240eb99f0b5faf77ece0e'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:47:12.100124 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-dbc834fe-c1c6-4198-ab03-d83316e2b92f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a4a4e810e8b240eb99f0b5faf77ece0e'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:47:12.112130 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1271/2557] 10.222.0.57 () {58 vars in 1223 bytes} [Fri Jul 1 16:47:11 2022] DELETE /identity/v3/users/a4a4e810e8b240eb99f0b5faf77ece0e => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 16:47:12.116733 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73eb47c1-446e-4d10-b8bb-1ba789c8c6ca None None] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:47:12.117219 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73eb47c1-446e-4d10-b8bb-1ba789c8c6ca None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:47:12.117326 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73eb47c1-446e-4d10-b8bb-1ba789c8c6ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:47:12.137043 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.auth.core [None req-73eb47c1-446e-4d10-b8bb-1ba789c8c6ca None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80312) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:47:12.138210 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-73eb47c1-446e-4d10-b8bb-1ba789c8c6ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:47:12.196908 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1287/2558] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 1 16:47:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 16:47:12.208382 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc834fe-c1c6-4198-ab03-d83316e2b92f admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:47:12.209814 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-dbc834fe-c1c6-4198-ab03-d83316e2b92f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:47:12.223886 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:47:12.271302 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc834fe-c1c6-4198-ab03-d83316e2b92f admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:47:12.275049 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d913d3b0-667b-4a99-b2ac-50bd2f0bbecd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ed200ce29684c5f80354fa9a8638fe2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0788e6725a84e57a4174284c1845755', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:47:12.275735 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d913d3b0-667b-4a99-b2ac-50bd2f0bbecd service neutron] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:47:12.276019 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d913d3b0-667b-4a99-b2ac-50bd2f0bbecd service neutron] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:47:12.276293 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d913d3b0-667b-4a99-b2ac-50bd2f0bbecd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:47:12.285791 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d913d3b0-667b-4a99-b2ac-50bd2f0bbecd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:47:12.287251 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d913d3b0-667b-4a99-b2ac-50bd2f0bbecd service neutron] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:47:12.297587 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1272/2559] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 1 16:47:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 16:47:12.574180 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73eb47c1-446e-4d10-b8bb-1ba789c8c6ca None None] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:47:12.575740 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-73eb47c1-446e-4d10-b8bb-1ba789c8c6ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:47:12.589999 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73eb47c1-446e-4d10-b8bb-1ba789c8c6ca None None] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:47:12.594883 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2efc5d4-ba7a-42bf-82e5-e617d3bd66f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:47:12.595777 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2efc5d4-ba7a-42bf-82e5-e617d3bd66f6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:47:12.596075 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2efc5d4-ba7a-42bf-82e5-e617d3bd66f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:47:12.597104 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2efc5d4-ba7a-42bf-82e5-e617d3bd66f6 admin admin] PATH_INFO: `/v3/projects/db98a6214dec4c0c9cbca95c8788171f` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:47:12.598367 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2efc5d4-ba7a-42bf-82e5-e617d3bd66f6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=db98a6214dec4c0c9cbca95c8788171f)` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:47:12.600384 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2efc5d4-ba7a-42bf-82e5-e617d3bd66f6 admin admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:47:12.624019 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:47:12.649932 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-d2efc5d4-ba7a-42bf-82e5-e617d3bd66f6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'db98a6214dec4c0c9cbca95c8788171f', 'request_id': 'req-d2efc5d4-ba7a-42bf-82e5-e617d3bd66f6'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:47:12.657019 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-d2efc5d4-ba7a-42bf-82e5-e617d3bd66f6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'db98a6214dec4c0c9cbca95c8788171f', 'request_id': 'req-d2efc5d4-ba7a-42bf-82e5-e617d3bd66f6'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:47:12.663964 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-d2efc5d4-ba7a-42bf-82e5-e617d3bd66f6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'db98a6214dec4c0c9cbca95c8788171f', 'request_id': 'req-d2efc5d4-ba7a-42bf-82e5-e617d3bd66f6'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:47:12.675163 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-d2efc5d4-ba7a-42bf-82e5-e617d3bd66f6 admin admin] The token cache is being invalidate because project db98a6214dec4c0c9cbca95c8788171f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80312) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 01 16:47:12.675454 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-d2efc5d4-ba7a-42bf-82e5-e617d3bd66f6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:47:12.676124 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-d2efc5d4-ba7a-42bf-82e5-e617d3bd66f6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:47:12.677678 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1288/2560] 10.222.0.57 () {58 vars in 1232 bytes} [Fri Jul 1 16:47:12 2022] DELETE /identity/v3/projects/db98a6214dec4c0c9cbca95c8788171f => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 16:47:12.682734 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681fb178-9d46-49e4-9083-2d2c65e0faca None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:47:12.683027 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681fb178-9d46-49e4-9083-2d2c65e0faca None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:47:12.683306 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681fb178-9d46-49e4-9083-2d2c65e0faca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:47:12.703099 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-681fb178-9d46-49e4-9083-2d2c65e0faca None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:47:12.704308 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-681fb178-9d46-49e4-9083-2d2c65e0faca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:47:12.772232 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1273/2561] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 1 16:47:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 16:47:12.780148 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc35ac5-b00b-460a-a579-47b3b28ada24 None None] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:47:12.780437 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc35ac5-b00b-460a-a579-47b3b28ada24 None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:47:12.780717 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc35ac5-b00b-460a-a579-47b3b28ada24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:47:12.799557 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.auth.core [None req-efc35ac5-b00b-460a-a579-47b3b28ada24 None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80312) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:47:12.800861 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-efc35ac5-b00b-460a-a579-47b3b28ada24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:47:12.808795 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1289/2562] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 1 16:47:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 16:47:12.812915 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681fb178-9d46-49e4-9083-2d2c65e0faca None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:47:12.822867 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681fb178-9d46-49e4-9083-2d2c65e0faca None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:47:12.827413 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3974190a-0194-42d2-b70a-9dcbb073ac3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:47:12.828066 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3974190a-0194-42d2-b70a-9dcbb073ac3b admin admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:47:12.828349 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3974190a-0194-42d2-b70a-9dcbb073ac3b admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:47:12.828627 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3974190a-0194-42d2-b70a-9dcbb073ac3b admin admin] PATH_INFO: `/v3/domains` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:47:12.829348 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3974190a-0194-42d2-b70a-9dcbb073ac3b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:47:12.829657 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3974190a-0194-42d2-b70a-9dcbb073ac3b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:47:12.831947 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3974190a-0194-42d2-b70a-9dcbb073ac3b admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:47:12.844038 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1274/2563] 10.222.0.57 () {58 vars in 1152 bytes} [Fri Jul 1 16:47:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:47:12.849007 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc35ac5-b00b-460a-a579-47b3b28ada24 None None] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:47:12.858186 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc35ac5-b00b-460a-a579-47b3b28ada24 None None] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:47:12.862894 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee922e6-163c-4536-a7ba-096de93e9eab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:47:12.863538 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee922e6-163c-4536-a7ba-096de93e9eab admin admin] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:47:12.863822 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee922e6-163c-4536-a7ba-096de93e9eab admin admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:47:12.864096 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee922e6-163c-4536-a7ba-096de93e9eab admin admin] PATH_INFO: `/v3/projects` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:47:12.864819 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee922e6-163c-4536-a7ba-096de93e9eab admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:47:12.866744 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee922e6-163c-4536-a7ba-096de93e9eab admin admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:47:12.892920 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1290/2564] 10.222.0.57 () {60 vars in 1152 bytes} [Fri Jul 1 16:47:12 2022] POST /identity/v3/projects => generated 383 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 16:47:12.897599 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3974190a-0194-42d2-b70a-9dcbb073ac3b admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:47:12.907776 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3974190a-0194-42d2-b70a-9dcbb073ac3b admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:47:12.964305 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd45354-6ed0-4a14-9728-224671833e70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:47:12.964959 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd45354-6ed0-4a14-9728-224671833e70 admin admin] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:47:12.965243 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd45354-6ed0-4a14-9728-224671833e70 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:47:12.965521 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd45354-6ed0-4a14-9728-224671833e70 admin admin] PATH_INFO: `/v3/users` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:47:12.966214 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd45354-6ed0-4a14-9728-224671833e70 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:47:12.968179 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd45354-6ed0-4a14-9728-224671833e70 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:47:12.988404 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1275/2565] 10.222.0.57 () {60 vars in 1143 bytes} [Fri Jul 1 16:47:12 2022] POST /identity/v3/users => generated 292 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 01 16:47:12.992580 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee922e6-163c-4536-a7ba-096de93e9eab admin admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:47:13.003276 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee922e6-163c-4536-a7ba-096de93e9eab admin admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:47:13.008149 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42594480-90aa-418a-8d3e-451796b3d7b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:47:13.008801 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42594480-90aa-418a-8d3e-451796b3d7b4 admin admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:47:13.009082 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42594480-90aa-418a-8d3e-451796b3d7b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:47:13.009357 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42594480-90aa-418a-8d3e-451796b3d7b4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:47:13.010104 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42594480-90aa-418a-8d3e-451796b3d7b4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:47:13.011736 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42594480-90aa-418a-8d3e-451796b3d7b4 admin admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:47:13.023376 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1291/2566] 10.222.0.57 () {58 vars in 1121 bytes} [Fri Jul 1 16:47:12 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 16:47:13.027520 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd45354-6ed0-4a14-9728-224671833e70 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:47:13.037891 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd45354-6ed0-4a14-9728-224671833e70 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:47:13.042710 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea37e95-5660-4e46-b18b-cb5f3972d3ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:47:13.043367 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea37e95-5660-4e46-b18b-cb5f3972d3ef admin admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:47:13.043655 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea37e95-5660-4e46-b18b-cb5f3972d3ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:47:13.043936 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea37e95-5660-4e46-b18b-cb5f3972d3ef admin admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:47:13.044668 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea37e95-5660-4e46-b18b-cb5f3972d3ef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:47:13.046260 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea37e95-5660-4e46-b18b-cb5f3972d3ef admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:47:13.058109 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1276/2567] 10.222.0.57 () {58 vars in 1121 bytes} [Fri Jul 1 16:47:13 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 16:47:13.062264 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42594480-90aa-418a-8d3e-451796b3d7b4 admin admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:47:13.073624 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42594480-90aa-418a-8d3e-451796b3d7b4 admin admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:47:13.078388 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8eb8e5a-d3d9-41ec-a51e-3b75fd9a07f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:47:13.079268 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8eb8e5a-d3d9-41ec-a51e-3b75fd9a07f6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:47:13.079557 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8eb8e5a-d3d9-41ec-a51e-3b75fd9a07f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:47:13.079840 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8eb8e5a-d3d9-41ec-a51e-3b75fd9a07f6 admin admin] PATH_INFO: `/v3/projects/5f7bb2b7df404fccb68dbd0cdc9ca702/users/893759e765464993955b9bc40103f526/roles/983be40bdab04c478eaa90c6b6a46b14` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:47:13.095165 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8eb8e5a-d3d9-41ec-a51e-3b75fd9a07f6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5f7bb2b7df404fccb68dbd0cdc9ca702, user_id=893759e765464993955b9bc40103f526, role_id=983be40bdab04c478eaa90c6b6a46b14)` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:47:13.098068 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8eb8e5a-d3d9-41ec-a51e-3b75fd9a07f6 admin admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:47:13.110607 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1292/2568] 10.222.0.57 () {60 vars in 1482 bytes} [Fri Jul 1 16:47:13 2022] PUT /identity/v3/projects/5f7bb2b7df404fccb68dbd0cdc9ca702/users/893759e765464993955b9bc40103f526/roles/983be40bdab04c478eaa90c6b6a46b14 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 16:47:13.120923 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f516d056-2f75-421a-b1a0-33c832281c21 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:47:13.121221 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f516d056-2f75-421a-b1a0-33c832281c21 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:47:13.121508 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f516d056-2f75-421a-b1a0-33c832281c21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:47:13.149052 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-f516d056-2f75-421a-b1a0-33c832281c21 None None] MFA Rules not processed for user `893759e765464993955b9bc40103f526`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:47:13.150278 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-f516d056-2f75-421a-b1a0-33c832281c21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:47:13.159652 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:47:13.217408 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1277/2569] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 1 16:47:13 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 16:47:13.222185 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8eb8e5a-d3d9-41ec-a51e-3b75fd9a07f6 admin admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:47:13.283975 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8eb8e5a-d3d9-41ec-a51e-3b75fd9a07f6 admin admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:47:13.288571 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8496f35-4cb1-4185-af29-3d8a2ac108b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:47:13.289233 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8496f35-4cb1-4185-af29-3d8a2ac108b5 admin admin] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:47:13.289515 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8496f35-4cb1-4185-af29-3d8a2ac108b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:47:13.289797 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8496f35-4cb1-4185-af29-3d8a2ac108b5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:47:13.290530 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8496f35-4cb1-4185-af29-3d8a2ac108b5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:47:13.292462 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8496f35-4cb1-4185-af29-3d8a2ac108b5 admin admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:47:13.317567 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1293/2570] 10.222.0.57 () {60 vars in 1152 bytes} [Fri Jul 1 16:47:13 2022] POST /identity/v3/projects => generated 387 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 16:47:13.321560 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f516d056-2f75-421a-b1a0-33c832281c21 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:47:13.333074 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f516d056-2f75-421a-b1a0-33c832281c21 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:47:13.390091 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbfa8b8-e7dd-4408-b4b0-71ba1c4cf100 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:47:13.390789 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbfa8b8-e7dd-4408-b4b0-71ba1c4cf100 admin admin] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:47:13.391076 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbfa8b8-e7dd-4408-b4b0-71ba1c4cf100 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:47:13.391355 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbfa8b8-e7dd-4408-b4b0-71ba1c4cf100 admin admin] PATH_INFO: `/v3/users` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:47:13.392042 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfbfa8b8-e7dd-4408-b4b0-71ba1c4cf100 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:47:13.394543 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfbfa8b8-e7dd-4408-b4b0-71ba1c4cf100 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:47:13.415071 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1278/2571] 10.222.0.57 () {60 vars in 1143 bytes} [Fri Jul 1 16:47:13 2022] POST /identity/v3/users => generated 294 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 16:47:13.419052 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8496f35-4cb1-4185-af29-3d8a2ac108b5 admin admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:47:13.430109 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8496f35-4cb1-4185-af29-3d8a2ac108b5 admin admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:47:13.434895 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e81a41-6aa5-43a6-b0f7-d07d66ace3ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:47:13.435555 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e81a41-6aa5-43a6-b0f7-d07d66ace3ca admin admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:47:13.435840 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e81a41-6aa5-43a6-b0f7-d07d66ace3ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:47:13.436121 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e81a41-6aa5-43a6-b0f7-d07d66ace3ca admin admin] PATH_INFO: `/v3/roles` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:47:13.436877 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e81a41-6aa5-43a6-b0f7-d07d66ace3ca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:47:13.438529 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e81a41-6aa5-43a6-b0f7-d07d66ace3ca admin admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:47:13.450113 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1294/2572] 10.222.0.57 () {58 vars in 1121 bytes} [Fri Jul 1 16:47:13 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 16:47:13.454033 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbfa8b8-e7dd-4408-b4b0-71ba1c4cf100 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:47:13.464542 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbfa8b8-e7dd-4408-b4b0-71ba1c4cf100 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:47:13.469401 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b1a6ff-79e5-4190-9335-2816ce9d6a3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:47:13.470059 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b1a6ff-79e5-4190-9335-2816ce9d6a3d admin admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:47:13.470363 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b1a6ff-79e5-4190-9335-2816ce9d6a3d admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:47:13.470644 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b1a6ff-79e5-4190-9335-2816ce9d6a3d admin admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:47:13.471474 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b1a6ff-79e5-4190-9335-2816ce9d6a3d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:47:13.473054 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b1a6ff-79e5-4190-9335-2816ce9d6a3d admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:47:13.482958 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1279/2573] 10.222.0.57 () {58 vars in 1121 bytes} [Fri Jul 1 16:47:13 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 16:47:13.487786 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e81a41-6aa5-43a6-b0f7-d07d66ace3ca admin admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:47:13.497807 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e81a41-6aa5-43a6-b0f7-d07d66ace3ca admin admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:47:13.502432 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b8a837-b135-4e71-9da3-47066f54bdb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:47:13.504036 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b8a837-b135-4e71-9da3-47066f54bdb9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:47:13.504404 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b8a837-b135-4e71-9da3-47066f54bdb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:47:13.504720 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b8a837-b135-4e71-9da3-47066f54bdb9 admin admin] PATH_INFO: `/v3/projects/1341bd2db7574fd8ab9e2680f2ad97aa/users/06bf9383537141bfa1c00c2ed00c3af8/roles/983be40bdab04c478eaa90c6b6a46b14` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:47:13.519394 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b8a837-b135-4e71-9da3-47066f54bdb9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1341bd2db7574fd8ab9e2680f2ad97aa, user_id=06bf9383537141bfa1c00c2ed00c3af8, role_id=983be40bdab04c478eaa90c6b6a46b14)` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:47:13.522050 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b8a837-b135-4e71-9da3-47066f54bdb9 admin admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:47:13.525444 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:DELETE=7,keystone:SELECT=88,keystone:INSERT=5 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:47:13.538754 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1295/2574] 10.222.0.57 () {60 vars in 1482 bytes} [Fri Jul 1 16:47:13 2022] PUT /identity/v3/projects/1341bd2db7574fd8ab9e2680f2ad97aa/users/06bf9383537141bfa1c00c2ed00c3af8/roles/983be40bdab04c478eaa90c6b6a46b14 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 16:47:13.550899 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f385059f-d0e4-4bbc-8746-b0b6884e56f0 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:47:13.551196 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f385059f-d0e4-4bbc-8746-b0b6884e56f0 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:47:13.551481 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f385059f-d0e4-4bbc-8746-b0b6884e56f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:47:13.578248 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-f385059f-d0e4-4bbc-8746-b0b6884e56f0 None None] MFA Rules not processed for user `06bf9383537141bfa1c00c2ed00c3af8`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:47:13.579492 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-f385059f-d0e4-4bbc-8746-b0b6884e56f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:47:13.645402 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1280/2575] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 1 16:47:13 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 16:47:13.649813 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b8a837-b135-4e71-9da3-47066f54bdb9 admin admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:47:13.716342 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b8a837-b135-4e71-9da3-47066f54bdb9 admin admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:47:13.721613 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6824cdef-6a9d-4058-b50f-1d4d95049e38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:47:13.722457 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6824cdef-6a9d-4058-b50f-1d4d95049e38 admin admin] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:47:13.722694 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6824cdef-6a9d-4058-b50f-1d4d95049e38 admin admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:47:13.723004 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6824cdef-6a9d-4058-b50f-1d4d95049e38 admin admin] PATH_INFO: `/v3/projects` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:47:13.723697 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6824cdef-6a9d-4058-b50f-1d4d95049e38 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:47:13.725599 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6824cdef-6a9d-4058-b50f-1d4d95049e38 admin admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:47:13.750704 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1296/2576] 10.222.0.57 () {60 vars in 1152 bytes} [Fri Jul 1 16:47:13 2022] POST /identity/v3/projects => generated 387 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 16:47:13.755035 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f385059f-d0e4-4bbc-8746-b0b6884e56f0 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:47:13.765817 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f385059f-d0e4-4bbc-8746-b0b6884e56f0 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:47:13.778761 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:47:13.823756 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5a6a1a-9e35-4ca3-a14e-8c31be47e56d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:47:13.824421 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5a6a1a-9e35-4ca3-a14e-8c31be47e56d admin admin] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:47:13.824703 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5a6a1a-9e35-4ca3-a14e-8c31be47e56d admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:47:13.824977 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5a6a1a-9e35-4ca3-a14e-8c31be47e56d admin admin] PATH_INFO: `/v3/users` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:47:13.825663 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f5a6a1a-9e35-4ca3-a14e-8c31be47e56d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:47:13.827610 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f5a6a1a-9e35-4ca3-a14e-8c31be47e56d admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:47:13.848453 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1281/2577] 10.222.0.57 () {60 vars in 1143 bytes} [Fri Jul 1 16:47:13 2022] POST /identity/v3/users => generated 300 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 16:47:13.852420 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6824cdef-6a9d-4058-b50f-1d4d95049e38 admin admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:47:13.863635 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6824cdef-6a9d-4058-b50f-1d4d95049e38 admin admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:47:13.868282 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efeba812-ef77-45a4-a342-55fcdd13ae0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:47:13.868943 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efeba812-ef77-45a4-a342-55fcdd13ae0f admin admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:47:13.869233 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efeba812-ef77-45a4-a342-55fcdd13ae0f admin admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:47:13.869714 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efeba812-ef77-45a4-a342-55fcdd13ae0f admin admin] PATH_INFO: `/v3/roles` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:47:13.870550 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efeba812-ef77-45a4-a342-55fcdd13ae0f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:47:13.872171 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efeba812-ef77-45a4-a342-55fcdd13ae0f admin admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:47:13.883810 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1297/2578] 10.222.0.57 () {58 vars in 1121 bytes} [Fri Jul 1 16:47:13 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 16:47:13.887903 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5a6a1a-9e35-4ca3-a14e-8c31be47e56d admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:47:13.898617 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5a6a1a-9e35-4ca3-a14e-8c31be47e56d admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:47:13.903267 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10bf08f-8bf1-4933-8717-5492e7216e53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:47:13.904141 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10bf08f-8bf1-4933-8717-5492e7216e53 admin admin] REQUEST_METHOD: `PUT` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:47:13.904427 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10bf08f-8bf1-4933-8717-5492e7216e53 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:47:13.904709 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10bf08f-8bf1-4933-8717-5492e7216e53 admin admin] PATH_INFO: `/v3/projects/a8eb26d109994c1089bb06573ab38811/users/ea2ec6fccb9043308cd1a9a2bba1e40f/roles/84ee739146934c1ab32c501ff8093644` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:47:13.920429 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d10bf08f-8bf1-4933-8717-5492e7216e53 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a8eb26d109994c1089bb06573ab38811, user_id=ea2ec6fccb9043308cd1a9a2bba1e40f, role_id=84ee739146934c1ab32c501ff8093644)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:47:13.923140 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d10bf08f-8bf1-4933-8717-5492e7216e53 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:47:13.935915 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1282/2579] 10.222.0.57 () {60 vars in 1482 bytes} [Fri Jul 1 16:47:13 2022] PUT /identity/v3/projects/a8eb26d109994c1089bb06573ab38811/users/ea2ec6fccb9043308cd1a9a2bba1e40f/roles/84ee739146934c1ab32c501ff8093644 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 16:47:13.939767 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efeba812-ef77-45a4-a342-55fcdd13ae0f admin admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:47:13.950917 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efeba812-ef77-45a4-a342-55fcdd13ae0f admin admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:47:14.006817 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8f107b-903b-40fd-ac86-4c1fe93f062f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:47:14.007481 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8f107b-903b-40fd-ac86-4c1fe93f062f admin admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:47:14.007766 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8f107b-903b-40fd-ac86-4c1fe93f062f admin admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:47:14.008045 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8f107b-903b-40fd-ac86-4c1fe93f062f admin admin] PATH_INFO: `/v3/roles` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:47:14.008772 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c8f107b-903b-40fd-ac86-4c1fe93f062f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:47:14.010393 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c8f107b-903b-40fd-ac86-4c1fe93f062f admin admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:47:14.020752 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1298/2580] 10.222.0.57 () {58 vars in 1121 bytes} [Fri Jul 1 16:47:13 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 16:47:14.024854 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10bf08f-8bf1-4933-8717-5492e7216e53 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:47:14.035445 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10bf08f-8bf1-4933-8717-5492e7216e53 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:47:14.040142 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5abc2ab6-b11e-4b13-aa72-9622994b1803 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:47:14.041016 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abc2ab6-b11e-4b13-aa72-9622994b1803 admin admin] REQUEST_METHOD: `PUT` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:47:14.041305 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abc2ab6-b11e-4b13-aa72-9622994b1803 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:47:14.041591 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abc2ab6-b11e-4b13-aa72-9622994b1803 admin admin] PATH_INFO: `/v3/projects/a8eb26d109994c1089bb06573ab38811/users/ea2ec6fccb9043308cd1a9a2bba1e40f/roles/84ee739146934c1ab32c501ff8093644` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:47:14.043562 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5abc2ab6-b11e-4b13-aa72-9622994b1803 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a8eb26d109994c1089bb06573ab38811, user_id=ea2ec6fccb9043308cd1a9a2bba1e40f, role_id=84ee739146934c1ab32c501ff8093644)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:47:14.046184 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5abc2ab6-b11e-4b13-aa72-9622994b1803 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:47:14.056717 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1283/2581] 10.222.0.57 () {60 vars in 1482 bytes} [Fri Jul 1 16:47:14 2022] PUT /identity/v3/projects/a8eb26d109994c1089bb06573ab38811/users/ea2ec6fccb9043308cd1a9a2bba1e40f/roles/84ee739146934c1ab32c501ff8093644 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 16:47:14.067867 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9c90f3-b7c3-4bd5-8007-aa0fd81f0bb1 None None] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:47:14.068160 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9c90f3-b7c3-4bd5-8007-aa0fd81f0bb1 None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:47:14.068441 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9c90f3-b7c3-4bd5-8007-aa0fd81f0bb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:47:14.094685 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.auth.core [None req-8f9c90f3-b7c3-4bd5-8007-aa0fd81f0bb1 None None] MFA Rules not processed for user `ea2ec6fccb9043308cd1a9a2bba1e40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80312) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:47:14.095878 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-8f9c90f3-b7c3-4bd5-8007-aa0fd81f0bb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:47:14.144933 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:47:14.163404 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1299/2582] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 1 16:47:14 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 16:47:14.174774 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5abc2ab6-b11e-4b13-aa72-9622994b1803 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:47:14.176628 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-5abc2ab6-b11e-4b13-aa72-9622994b1803 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:47:14.235826 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5abc2ab6-b11e-4b13-aa72-9622994b1803 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:47:14.240184 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a9776e-1502-40ff-ba77-f22d98ece2f8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbddf971a2354bbdbb9da1d63f3abf88', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0788e6725a84e57a4174284c1845755', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:47:14.240874 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a9776e-1502-40ff-ba77-f22d98ece2f8 service cinder] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:47:14.241146 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a9776e-1502-40ff-ba77-f22d98ece2f8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:47:14.241426 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a9776e-1502-40ff-ba77-f22d98ece2f8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:47:14.250451 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a9776e-1502-40ff-ba77-f22d98ece2f8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:47:14.251887 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a9776e-1502-40ff-ba77-f22d98ece2f8 service cinder] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:47:14.262114 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1284/2583] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 1 16:47:14 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 16:47:14.766917 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9c90f3-b7c3-4bd5-8007-aa0fd81f0bb1 None None] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:47:14.768443 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-8f9c90f3-b7c3-4bd5-8007-aa0fd81f0bb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:47:14.780637 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9c90f3-b7c3-4bd5-8007-aa0fd81f0bb1 None None] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:47:14.784925 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5473529c-0184-4625-934f-3298fec006dd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbddf971a2354bbdbb9da1d63f3abf88', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0788e6725a84e57a4174284c1845755', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:47:14.785887 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5473529c-0184-4625-934f-3298fec006dd service cinder] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:47:14.786309 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5473529c-0184-4625-934f-3298fec006dd service cinder] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:47:14.786776 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5473529c-0184-4625-934f-3298fec006dd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:47:14.844111 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5473529c-0184-4625-934f-3298fec006dd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:47:14.845588 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5473529c-0184-4625-934f-3298fec006dd service cinder] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:47:14.855716 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1300/2584] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 1 16:47:14 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 16:47:18.841463 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a9776e-1502-40ff-ba77-f22d98ece2f8 service cinder] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:47:18.851458 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a9776e-1502-40ff-ba77-f22d98ece2f8 service cinder] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:47:18.855156 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eeeeeda-e8be-4af8-8f40-1b36adf4ea3f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbddf971a2354bbdbb9da1d63f3abf88', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0788e6725a84e57a4174284c1845755', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:47:18.855802 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eeeeeda-e8be-4af8-8f40-1b36adf4ea3f service cinder] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:47:18.856089 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eeeeeda-e8be-4af8-8f40-1b36adf4ea3f service cinder] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:47:18.856371 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eeeeeda-e8be-4af8-8f40-1b36adf4ea3f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:47:18.912996 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eeeeeda-e8be-4af8-8f40-1b36adf4ea3f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:47:18.914374 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eeeeeda-e8be-4af8-8f40-1b36adf4ea3f service cinder] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:47:18.923342 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1285/2585] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 1 16:47:18 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 16:47:20.983755 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5473529c-0184-4625-934f-3298fec006dd service cinder] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:47:21.045842 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5473529c-0184-4625-934f-3298fec006dd service cinder] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:47:21.051332 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6dd11e3-fc1d-423e-bc1f-f5d18f4fd5ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:47:21.052141 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6dd11e3-fc1d-423e-bc1f-f5d18f4fd5ef admin admin] REQUEST_METHOD: `DELETE` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:47:21.052427 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6dd11e3-fc1d-423e-bc1f-f5d18f4fd5ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:47:21.052703 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6dd11e3-fc1d-423e-bc1f-f5d18f4fd5ef admin admin] PATH_INFO: `/v3/users/893759e765464993955b9bc40103f526` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:47:21.053876 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6dd11e3-fc1d-423e-bc1f-f5d18f4fd5ef admin admin] RBAC: Authorizing `identity:delete_user(user_id=893759e765464993955b9bc40103f526)` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:47:21.055982 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6dd11e3-fc1d-423e-bc1f-f5d18f4fd5ef admin admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:47:21.103011 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-d6dd11e3-fc1d-423e-bc1f-f5d18f4fd5ef admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '893759e765464993955b9bc40103f526'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:47:21.116833 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-d6dd11e3-fc1d-423e-bc1f-f5d18f4fd5ef admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '893759e765464993955b9bc40103f526'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:47:21.123535 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-d6dd11e3-fc1d-423e-bc1f-f5d18f4fd5ef admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '893759e765464993955b9bc40103f526'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:47:21.124222 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-d6dd11e3-fc1d-423e-bc1f-f5d18f4fd5ef admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '893759e765464993955b9bc40103f526'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:47:21.124821 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-d6dd11e3-fc1d-423e-bc1f-f5d18f4fd5ef admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '893759e765464993955b9bc40103f526'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:47:21.137936 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1301/2586] 10.222.0.57 () {58 vars in 1223 bytes} [Fri Jul 1 16:47:20 2022] DELETE /identity/v3/users/893759e765464993955b9bc40103f526 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 16:47:21.142559 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa562b03-a3e8-4d5d-84c6-7b04b172f68c None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:47:21.143148 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa562b03-a3e8-4d5d-84c6-7b04b172f68c None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:47:21.143255 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa562b03-a3e8-4d5d-84c6-7b04b172f68c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:47:21.160306 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-aa562b03-a3e8-4d5d-84c6-7b04b172f68c None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:47:21.161525 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-aa562b03-a3e8-4d5d-84c6-7b04b172f68c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:47:21.166445 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:47:21.220407 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1286/2587] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 1 16:47:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 16:47:21.230696 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6dd11e3-fc1d-423e-bc1f-f5d18f4fd5ef admin admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:47:21.232142 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-d6dd11e3-fc1d-423e-bc1f-f5d18f4fd5ef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:47:21.246107 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:47:21.294796 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6dd11e3-fc1d-423e-bc1f-f5d18f4fd5ef admin admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:47:21.298399 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f557f34-de4e-40e0-ba60-4d1477e3ba4c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ed200ce29684c5f80354fa9a8638fe2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0788e6725a84e57a4174284c1845755', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:47:21.299192 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f557f34-de4e-40e0-ba60-4d1477e3ba4c service neutron] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:47:21.299792 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f557f34-de4e-40e0-ba60-4d1477e3ba4c service neutron] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:47:21.299792 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f557f34-de4e-40e0-ba60-4d1477e3ba4c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:47:21.308623 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f557f34-de4e-40e0-ba60-4d1477e3ba4c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:47:21.309979 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f557f34-de4e-40e0-ba60-4d1477e3ba4c service neutron] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:47:21.320845 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1302/2588] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 1 16:47:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 16:47:21.599059 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa562b03-a3e8-4d5d-84c6-7b04b172f68c None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:47:21.601026 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-aa562b03-a3e8-4d5d-84c6-7b04b172f68c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:47:21.614520 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa562b03-a3e8-4d5d-84c6-7b04b172f68c None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:47:21.619218 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a012cf6-a2cd-4e09-ac2b-4f0a63dfd738 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:47:21.620078 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a012cf6-a2cd-4e09-ac2b-4f0a63dfd738 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:47:21.620370 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a012cf6-a2cd-4e09-ac2b-4f0a63dfd738 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:47:21.620648 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a012cf6-a2cd-4e09-ac2b-4f0a63dfd738 admin admin] PATH_INFO: `/v3/projects/5f7bb2b7df404fccb68dbd0cdc9ca702` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:47:21.621849 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a012cf6-a2cd-4e09-ac2b-4f0a63dfd738 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5f7bb2b7df404fccb68dbd0cdc9ca702)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:47:21.623795 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a012cf6-a2cd-4e09-ac2b-4f0a63dfd738 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:47:21.680666 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-8a012cf6-a2cd-4e09-ac2b-4f0a63dfd738 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5f7bb2b7df404fccb68dbd0cdc9ca702', 'request_id': 'req-8a012cf6-a2cd-4e09-ac2b-4f0a63dfd738'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:47:21.686085 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-8a012cf6-a2cd-4e09-ac2b-4f0a63dfd738 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5f7bb2b7df404fccb68dbd0cdc9ca702', 'request_id': 'req-8a012cf6-a2cd-4e09-ac2b-4f0a63dfd738'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:47:21.692440 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-8a012cf6-a2cd-4e09-ac2b-4f0a63dfd738 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5f7bb2b7df404fccb68dbd0cdc9ca702', 'request_id': 'req-8a012cf6-a2cd-4e09-ac2b-4f0a63dfd738'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:47:21.704483 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-8a012cf6-a2cd-4e09-ac2b-4f0a63dfd738 admin admin] The token cache is being invalidate because project 5f7bb2b7df404fccb68dbd0cdc9ca702 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80311) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 01 16:47:21.704814 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-8a012cf6-a2cd-4e09-ac2b-4f0a63dfd738 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:47:21.705936 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-8a012cf6-a2cd-4e09-ac2b-4f0a63dfd738 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:47:21.708078 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1287/2589] 10.222.0.57 () {58 vars in 1232 bytes} [Fri Jul 1 16:47:21 2022] DELETE /identity/v3/projects/5f7bb2b7df404fccb68dbd0cdc9ca702 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 16:47:21.712886 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f557f34-de4e-40e0-ba60-4d1477e3ba4c service neutron] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:47:21.726757 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f557f34-de4e-40e0-ba60-4d1477e3ba4c service neutron] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:47:21.787906 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0570ab-9d56-45d5-a83a-6a9b1d3f125a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:47:21.788768 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0570ab-9d56-45d5-a83a-6a9b1d3f125a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:47:21.789049 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0570ab-9d56-45d5-a83a-6a9b1d3f125a admin admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:47:21.789322 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0570ab-9d56-45d5-a83a-6a9b1d3f125a admin admin] PATH_INFO: `/v3/users/06bf9383537141bfa1c00c2ed00c3af8` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:47:21.790528 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd0570ab-9d56-45d5-a83a-6a9b1d3f125a admin admin] RBAC: Authorizing `identity:delete_user(user_id=06bf9383537141bfa1c00c2ed00c3af8)` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:47:21.792475 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd0570ab-9d56-45d5-a83a-6a9b1d3f125a admin admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:47:21.837456 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-fd0570ab-9d56-45d5-a83a-6a9b1d3f125a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '06bf9383537141bfa1c00c2ed00c3af8'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:47:21.850642 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-fd0570ab-9d56-45d5-a83a-6a9b1d3f125a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '06bf9383537141bfa1c00c2ed00c3af8'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:47:21.856953 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-fd0570ab-9d56-45d5-a83a-6a9b1d3f125a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '06bf9383537141bfa1c00c2ed00c3af8'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:47:21.857651 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-fd0570ab-9d56-45d5-a83a-6a9b1d3f125a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '06bf9383537141bfa1c00c2ed00c3af8'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:47:21.858242 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-fd0570ab-9d56-45d5-a83a-6a9b1d3f125a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '06bf9383537141bfa1c00c2ed00c3af8'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:47:21.871880 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1303/2590] 10.222.0.57 () {58 vars in 1223 bytes} [Fri Jul 1 16:47:21 2022] DELETE /identity/v3/users/06bf9383537141bfa1c00c2ed00c3af8 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 16:47:22.165737 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a012cf6-a2cd-4e09-ac2b-4f0a63dfd738 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:47:22.167321 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-8a012cf6-a2cd-4e09-ac2b-4f0a63dfd738 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:47:22.181889 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a012cf6-a2cd-4e09-ac2b-4f0a63dfd738 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:47:22.240347 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646cc882-0db2-4c14-8052-3a1e63f7b0e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:47:22.241210 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646cc882-0db2-4c14-8052-3a1e63f7b0e2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:47:22.241499 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646cc882-0db2-4c14-8052-3a1e63f7b0e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:47:22.241779 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646cc882-0db2-4c14-8052-3a1e63f7b0e2 admin admin] PATH_INFO: `/v3/projects/1341bd2db7574fd8ab9e2680f2ad97aa` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:47:22.242989 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-646cc882-0db2-4c14-8052-3a1e63f7b0e2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1341bd2db7574fd8ab9e2680f2ad97aa)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:47:22.244903 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-646cc882-0db2-4c14-8052-3a1e63f7b0e2 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:47:22.282297 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:47:22.292071 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-646cc882-0db2-4c14-8052-3a1e63f7b0e2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1341bd2db7574fd8ab9e2680f2ad97aa', 'request_id': 'req-646cc882-0db2-4c14-8052-3a1e63f7b0e2'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:47:22.297626 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-646cc882-0db2-4c14-8052-3a1e63f7b0e2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1341bd2db7574fd8ab9e2680f2ad97aa', 'request_id': 'req-646cc882-0db2-4c14-8052-3a1e63f7b0e2'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:47:22.304003 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-646cc882-0db2-4c14-8052-3a1e63f7b0e2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1341bd2db7574fd8ab9e2680f2ad97aa', 'request_id': 'req-646cc882-0db2-4c14-8052-3a1e63f7b0e2'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:47:22.316151 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-646cc882-0db2-4c14-8052-3a1e63f7b0e2 admin admin] The token cache is being invalidate because project 1341bd2db7574fd8ab9e2680f2ad97aa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80311) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 01 16:47:22.316433 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-646cc882-0db2-4c14-8052-3a1e63f7b0e2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:47:22.317048 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-646cc882-0db2-4c14-8052-3a1e63f7b0e2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:47:22.318641 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1288/2591] 10.222.0.57 () {58 vars in 1232 bytes} [Fri Jul 1 16:47:22 2022] DELETE /identity/v3/projects/1341bd2db7574fd8ab9e2680f2ad97aa => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 16:47:22.322722 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0570ab-9d56-45d5-a83a-6a9b1d3f125a admin admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:47:22.324492 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-fd0570ab-9d56-45d5-a83a-6a9b1d3f125a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:47:22.338040 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0570ab-9d56-45d5-a83a-6a9b1d3f125a admin admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:47:22.367147 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:47:22.396154 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f38e0c-bb86-4da9-b04c-b02d892b988c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:47:22.397023 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f38e0c-bb86-4da9-b04c-b02d892b988c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:47:22.397502 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f38e0c-bb86-4da9-b04c-b02d892b988c admin admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:47:22.397661 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f38e0c-bb86-4da9-b04c-b02d892b988c admin admin] PATH_INFO: `/v3/users/ea2ec6fccb9043308cd1a9a2bba1e40f` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:47:22.398784 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f38e0c-bb86-4da9-b04c-b02d892b988c admin admin] RBAC: Authorizing `identity:delete_user(user_id=ea2ec6fccb9043308cd1a9a2bba1e40f)` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:47:22.400698 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f38e0c-bb86-4da9-b04c-b02d892b988c admin admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:47:22.445683 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-b9f38e0c-bb86-4da9-b04c-b02d892b988c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ea2ec6fccb9043308cd1a9a2bba1e40f'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:47:22.459609 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-b9f38e0c-bb86-4da9-b04c-b02d892b988c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ea2ec6fccb9043308cd1a9a2bba1e40f'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:47:22.466449 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-b9f38e0c-bb86-4da9-b04c-b02d892b988c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ea2ec6fccb9043308cd1a9a2bba1e40f'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:47:22.467142 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-b9f38e0c-bb86-4da9-b04c-b02d892b988c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ea2ec6fccb9043308cd1a9a2bba1e40f'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:47:22.467754 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-b9f38e0c-bb86-4da9-b04c-b02d892b988c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ea2ec6fccb9043308cd1a9a2bba1e40f'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:47:22.481284 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1304/2592] 10.222.0.57 () {58 vars in 1223 bytes} [Fri Jul 1 16:47:22 2022] DELETE /identity/v3/users/ea2ec6fccb9043308cd1a9a2bba1e40f => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 16:47:22.761351 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646cc882-0db2-4c14-8052-3a1e63f7b0e2 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:47:22.762882 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-646cc882-0db2-4c14-8052-3a1e63f7b0e2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:47:22.777877 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646cc882-0db2-4c14-8052-3a1e63f7b0e2 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:47:22.836995 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763f0c8c-e419-46e9-900a-08f9ec720beb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:47:22.837861 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763f0c8c-e419-46e9-900a-08f9ec720beb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:47:22.838143 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763f0c8c-e419-46e9-900a-08f9ec720beb admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:47:22.838448 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763f0c8c-e419-46e9-900a-08f9ec720beb admin admin] PATH_INFO: `/v3/projects/a8eb26d109994c1089bb06573ab38811` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:47:22.839616 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-763f0c8c-e419-46e9-900a-08f9ec720beb admin admin] RBAC: Authorizing `identity:delete_project(project_id=a8eb26d109994c1089bb06573ab38811)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:47:22.841573 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-763f0c8c-e419-46e9-900a-08f9ec720beb admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:47:22.889677 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-763f0c8c-e419-46e9-900a-08f9ec720beb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a8eb26d109994c1089bb06573ab38811', 'request_id': 'req-763f0c8c-e419-46e9-900a-08f9ec720beb'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:47:22.895038 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-763f0c8c-e419-46e9-900a-08f9ec720beb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a8eb26d109994c1089bb06573ab38811', 'request_id': 'req-763f0c8c-e419-46e9-900a-08f9ec720beb'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:47:22.900983 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-763f0c8c-e419-46e9-900a-08f9ec720beb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a8eb26d109994c1089bb06573ab38811', 'request_id': 'req-763f0c8c-e419-46e9-900a-08f9ec720beb'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:47:22.914019 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-763f0c8c-e419-46e9-900a-08f9ec720beb admin admin] The token cache is being invalidate because project a8eb26d109994c1089bb06573ab38811 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80311) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 01 16:47:22.914311 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-763f0c8c-e419-46e9-900a-08f9ec720beb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:47:22.914995 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-763f0c8c-e419-46e9-900a-08f9ec720beb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:47:22.916665 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1289/2593] 10.222.0.57 () {58 vars in 1232 bytes} [Fri Jul 1 16:47:22 2022] DELETE /identity/v3/projects/a8eb26d109994c1089bb06573ab38811 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 16:47:22.923003 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1d1df2-c0fa-43c4-a585-0e05bacafe58 None None] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:47:22.923295 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1d1df2-c0fa-43c4-a585-0e05bacafe58 None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:47:22.923573 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1d1df2-c0fa-43c4-a585-0e05bacafe58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:47:22.943286 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.auth.core [None req-ae1d1df2-c0fa-43c4-a585-0e05bacafe58 None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80312) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:47:22.944499 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-ae1d1df2-c0fa-43c4-a585-0e05bacafe58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:47:23.004310 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1305/2594] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 1 16:47:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 16:47:23.013414 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962f74a2-70bb-41f0-a7e9-56f7be2e8bc3 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:47:23.013707 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962f74a2-70bb-41f0-a7e9-56f7be2e8bc3 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:47:23.013992 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962f74a2-70bb-41f0-a7e9-56f7be2e8bc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:47:23.032744 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-962f74a2-70bb-41f0-a7e9-56f7be2e8bc3 None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:47:23.034469 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-962f74a2-70bb-41f0-a7e9-56f7be2e8bc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:47:23.042416 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1290/2595] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 1 16:47:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 16:47:23.046568 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1d1df2-c0fa-43c4-a585-0e05bacafe58 None None] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:47:23.059062 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1d1df2-c0fa-43c4-a585-0e05bacafe58 None None] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:47:23.064214 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904a6354-62d8-4e44-ae05-97f204c88543 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:47:23.065451 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904a6354-62d8-4e44-ae05-97f204c88543 admin admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:47:23.065735 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904a6354-62d8-4e44-ae05-97f204c88543 admin admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:47:23.066012 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904a6354-62d8-4e44-ae05-97f204c88543 admin admin] PATH_INFO: `/v3/domains` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:47:23.066784 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904a6354-62d8-4e44-ae05-97f204c88543 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80312) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:47:23.067094 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904a6354-62d8-4e44-ae05-97f204c88543 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:47:23.068700 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904a6354-62d8-4e44-ae05-97f204c88543 admin admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:47:23.081378 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1306/2596] 10.222.0.57 () {58 vars in 1152 bytes} [Fri Jul 1 16:47:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:47:23.086122 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962f74a2-70bb-41f0-a7e9-56f7be2e8bc3 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:47:23.096509 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962f74a2-70bb-41f0-a7e9-56f7be2e8bc3 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:47:23.101203 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03aeeb71-e082-4c1d-8724-fc0e13465806 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:47:23.101854 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03aeeb71-e082-4c1d-8724-fc0e13465806 admin admin] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:47:23.102137 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03aeeb71-e082-4c1d-8724-fc0e13465806 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:47:23.102442 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03aeeb71-e082-4c1d-8724-fc0e13465806 admin admin] PATH_INFO: `/v3/projects` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:47:23.103254 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03aeeb71-e082-4c1d-8724-fc0e13465806 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:47:23.105033 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03aeeb71-e082-4c1d-8724-fc0e13465806 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:47:23.130870 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1291/2597] 10.222.0.57 () {60 vars in 1152 bytes} [Fri Jul 1 16:47:23 2022] POST /identity/v3/projects => generated 391 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 01 16:47:23.135213 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904a6354-62d8-4e44-ae05-97f204c88543 admin admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:47:23.146388 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904a6354-62d8-4e44-ae05-97f204c88543 admin admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:47:23.179610 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:47:23.203828 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a31e77a-4247-4394-ad36-972668a6163a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:47:23.204503 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a31e77a-4247-4394-ad36-972668a6163a admin admin] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:47:23.204792 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a31e77a-4247-4394-ad36-972668a6163a admin admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:47:23.205642 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a31e77a-4247-4394-ad36-972668a6163a admin admin] PATH_INFO: `/v3/users` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:47:23.206417 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a31e77a-4247-4394-ad36-972668a6163a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:47:23.208592 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a31e77a-4247-4394-ad36-972668a6163a admin admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:47:23.228606 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1307/2598] 10.222.0.57 () {60 vars in 1143 bytes} [Fri Jul 1 16:47:23 2022] POST /identity/v3/users => generated 296 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 16:47:23.232650 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03aeeb71-e082-4c1d-8724-fc0e13465806 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:47:23.242943 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03aeeb71-e082-4c1d-8724-fc0e13465806 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:47:23.249516 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41cab9c-8a7f-42b0-9efa-8fb72e15b792 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:47:23.250188 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41cab9c-8a7f-42b0-9efa-8fb72e15b792 admin admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:47:23.250529 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41cab9c-8a7f-42b0-9efa-8fb72e15b792 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:47:23.250812 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41cab9c-8a7f-42b0-9efa-8fb72e15b792 admin admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:47:23.251550 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a41cab9c-8a7f-42b0-9efa-8fb72e15b792 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:47:23.254272 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a41cab9c-8a7f-42b0-9efa-8fb72e15b792 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:47:23.265540 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1292/2599] 10.222.0.57 () {58 vars in 1121 bytes} [Fri Jul 1 16:47:23 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 16:47:23.271052 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a31e77a-4247-4394-ad36-972668a6163a admin admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:47:23.281157 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a31e77a-4247-4394-ad36-972668a6163a admin admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:47:23.286770 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96cfeacc-21b2-469f-a35f-962f0e34c49e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:47:23.287453 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cfeacc-21b2-469f-a35f-962f0e34c49e admin admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:47:23.287733 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cfeacc-21b2-469f-a35f-962f0e34c49e admin admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:47:23.288011 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cfeacc-21b2-469f-a35f-962f0e34c49e admin admin] PATH_INFO: `/v3/roles` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:47:23.288738 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96cfeacc-21b2-469f-a35f-962f0e34c49e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:47:23.290319 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96cfeacc-21b2-469f-a35f-962f0e34c49e admin admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:47:23.300291 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1308/2600] 10.222.0.57 () {58 vars in 1121 bytes} [Fri Jul 1 16:47:23 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 16:47:23.304402 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41cab9c-8a7f-42b0-9efa-8fb72e15b792 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:47:23.314765 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41cab9c-8a7f-42b0-9efa-8fb72e15b792 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:47:23.319458 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c9873b-4bc6-4286-8d63-f1c64b1bea3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:47:23.320336 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c9873b-4bc6-4286-8d63-f1c64b1bea3f admin admin] REQUEST_METHOD: `PUT` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:47:23.320620 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c9873b-4bc6-4286-8d63-f1c64b1bea3f admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:47:23.320901 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c9873b-4bc6-4286-8d63-f1c64b1bea3f admin admin] PATH_INFO: `/v3/projects/08835d4a4bc049b7b4e6c3476a306a47/users/6031243ff03b493b9d7d9587a5571595/roles/983be40bdab04c478eaa90c6b6a46b14` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:47:23.334724 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=86,keystone:DELETE=11,keystone:INSERT=3 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:47:23.338182 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c9873b-4bc6-4286-8d63-f1c64b1bea3f admin admin] RBAC: Authorizing `identity:create_grant(project_id=08835d4a4bc049b7b4e6c3476a306a47, user_id=6031243ff03b493b9d7d9587a5571595, role_id=983be40bdab04c478eaa90c6b6a46b14)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:47:23.341047 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c9873b-4bc6-4286-8d63-f1c64b1bea3f admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:47:23.361068 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1293/2601] 10.222.0.57 () {60 vars in 1482 bytes} [Fri Jul 1 16:47:23 2022] PUT /identity/v3/projects/08835d4a4bc049b7b4e6c3476a306a47/users/6031243ff03b493b9d7d9587a5571595/roles/983be40bdab04c478eaa90c6b6a46b14 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 16:47:23.373677 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805494f3-3130-49a9-bf6f-c3318b5cdd1e None None] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:47:23.374193 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805494f3-3130-49a9-bf6f-c3318b5cdd1e None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:47:23.374193 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805494f3-3130-49a9-bf6f-c3318b5cdd1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:47:23.402646 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.auth.core [None req-805494f3-3130-49a9-bf6f-c3318b5cdd1e None None] MFA Rules not processed for user `6031243ff03b493b9d7d9587a5571595`. Rule list: `[]` (Enabled: `True`). {{(pid=80312) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:47:23.403779 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-805494f3-3130-49a9-bf6f-c3318b5cdd1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:47:23.469273 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1309/2602] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 1 16:47:23 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 16:47:23.474545 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c9873b-4bc6-4286-8d63-f1c64b1bea3f admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:47:23.536636 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c9873b-4bc6-4286-8d63-f1c64b1bea3f admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:47:23.541373 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287e42cd-57b8-49fe-9d0f-a46d8dd89f23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:47:23.542029 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287e42cd-57b8-49fe-9d0f-a46d8dd89f23 admin admin] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:47:23.542313 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287e42cd-57b8-49fe-9d0f-a46d8dd89f23 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:47:23.542629 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287e42cd-57b8-49fe-9d0f-a46d8dd89f23 admin admin] PATH_INFO: `/v3/projects` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:47:23.543319 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-287e42cd-57b8-49fe-9d0f-a46d8dd89f23 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:47:23.545222 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-287e42cd-57b8-49fe-9d0f-a46d8dd89f23 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:47:23.569977 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1294/2603] 10.222.0.57 () {60 vars in 1152 bytes} [Fri Jul 1 16:47:23 2022] POST /identity/v3/projects => generated 391 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 16:47:23.574007 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805494f3-3130-49a9-bf6f-c3318b5cdd1e None None] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:47:23.585816 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805494f3-3130-49a9-bf6f-c3318b5cdd1e None None] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:47:23.643587 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e10dd7-5902-4895-87f7-3bb33e2028cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:47:23.644636 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e10dd7-5902-4895-87f7-3bb33e2028cc admin admin] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:47:23.644853 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e10dd7-5902-4895-87f7-3bb33e2028cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:47:23.645136 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e10dd7-5902-4895-87f7-3bb33e2028cc admin admin] PATH_INFO: `/v3/users` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:47:23.649368 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e10dd7-5902-4895-87f7-3bb33e2028cc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:47:23.653254 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e10dd7-5902-4895-87f7-3bb33e2028cc admin admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:47:23.676858 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1310/2604] 10.222.0.57 () {60 vars in 1143 bytes} [Fri Jul 1 16:47:23 2022] POST /identity/v3/users => generated 302 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 16:47:23.678215 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287e42cd-57b8-49fe-9d0f-a46d8dd89f23 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:47:23.688803 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287e42cd-57b8-49fe-9d0f-a46d8dd89f23 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:47:23.694411 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d862242-a9dd-40d4-91ff-76249f48db0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:47:23.695176 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d862242-a9dd-40d4-91ff-76249f48db0b admin admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:47:23.695646 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d862242-a9dd-40d4-91ff-76249f48db0b admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:47:23.695935 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d862242-a9dd-40d4-91ff-76249f48db0b admin admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:47:23.696683 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d862242-a9dd-40d4-91ff-76249f48db0b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:47:23.698285 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d862242-a9dd-40d4-91ff-76249f48db0b admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:47:23.711083 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1295/2605] 10.222.0.57 () {58 vars in 1121 bytes} [Fri Jul 1 16:47:23 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 16:47:23.715360 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e10dd7-5902-4895-87f7-3bb33e2028cc admin admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:47:23.725472 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e10dd7-5902-4895-87f7-3bb33e2028cc admin admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:47:23.730867 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6344f4-2a3b-4b07-a472-7a3618c7fdb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:47:23.731743 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6344f4-2a3b-4b07-a472-7a3618c7fdb5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:47:23.732023 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6344f4-2a3b-4b07-a472-7a3618c7fdb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:47:23.732323 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6344f4-2a3b-4b07-a472-7a3618c7fdb5 admin admin] PATH_INFO: `/v3/projects/309d7e719187462c87649aaf1ac30679/users/6183c7d41fa04c90b72382c306640817/roles/84ee739146934c1ab32c501ff8093644` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:47:23.747529 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b6344f4-2a3b-4b07-a472-7a3618c7fdb5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=309d7e719187462c87649aaf1ac30679, user_id=6183c7d41fa04c90b72382c306640817, role_id=84ee739146934c1ab32c501ff8093644)` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:47:23.750276 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b6344f4-2a3b-4b07-a472-7a3618c7fdb5 admin admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:47:23.764066 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1311/2606] 10.222.0.57 () {60 vars in 1482 bytes} [Fri Jul 1 16:47:23 2022] PUT /identity/v3/projects/309d7e719187462c87649aaf1ac30679/users/6183c7d41fa04c90b72382c306640817/roles/84ee739146934c1ab32c501ff8093644 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 16:47:23.768568 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d862242-a9dd-40d4-91ff-76249f48db0b admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:47:23.780805 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d862242-a9dd-40d4-91ff-76249f48db0b admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:47:23.841492 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b58d00-42ab-4328-a1b4-7c1fd1da4fa5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:47:23.842165 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b58d00-42ab-4328-a1b4-7c1fd1da4fa5 admin admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:47:23.842482 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b58d00-42ab-4328-a1b4-7c1fd1da4fa5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:47:23.842767 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b58d00-42ab-4328-a1b4-7c1fd1da4fa5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:47:23.843543 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b58d00-42ab-4328-a1b4-7c1fd1da4fa5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:47:23.845137 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b58d00-42ab-4328-a1b4-7c1fd1da4fa5 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:47:23.855747 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1296/2607] 10.222.0.57 () {58 vars in 1121 bytes} [Fri Jul 1 16:47:23 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 16:47:23.859830 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6344f4-2a3b-4b07-a472-7a3618c7fdb5 admin admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:47:23.871189 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6344f4-2a3b-4b07-a472-7a3618c7fdb5 admin admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:47:23.876020 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3cac394-999e-4b1b-93f6-9d413db614c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:47:23.877630 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cac394-999e-4b1b-93f6-9d413db614c9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:47:23.877927 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cac394-999e-4b1b-93f6-9d413db614c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:47:23.878204 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cac394-999e-4b1b-93f6-9d413db614c9 admin admin] PATH_INFO: `/v3/projects/309d7e719187462c87649aaf1ac30679/users/6183c7d41fa04c90b72382c306640817/roles/84ee739146934c1ab32c501ff8093644` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:47:23.880229 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3cac394-999e-4b1b-93f6-9d413db614c9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=309d7e719187462c87649aaf1ac30679, user_id=6183c7d41fa04c90b72382c306640817, role_id=84ee739146934c1ab32c501ff8093644)` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:47:23.882912 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3cac394-999e-4b1b-93f6-9d413db614c9 admin admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:47:23.888381 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:47:23.895364 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1312/2608] 10.222.0.57 () {60 vars in 1482 bytes} [Fri Jul 1 16:47:23 2022] PUT /identity/v3/projects/309d7e719187462c87649aaf1ac30679/users/6183c7d41fa04c90b72382c306640817/roles/84ee739146934c1ab32c501ff8093644 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 16:47:23.906468 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf5f1f4-44bf-4ae1-8b67-c701a5407c10 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:47:23.906702 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf5f1f4-44bf-4ae1-8b67-c701a5407c10 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:47:23.906986 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf5f1f4-44bf-4ae1-8b67-c701a5407c10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:47:23.935466 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-fbf5f1f4-44bf-4ae1-8b67-c701a5407c10 None None] MFA Rules not processed for user `6183c7d41fa04c90b72382c306640817`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:47:23.936682 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-fbf5f1f4-44bf-4ae1-8b67-c701a5407c10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:47:23.983058 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:47:24.006703 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1297/2609] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 1 16:47:23 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 16:47:24.017394 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3cac394-999e-4b1b-93f6-9d413db614c9 admin admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:47:24.019549 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-b3cac394-999e-4b1b-93f6-9d413db614c9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:47:24.078219 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3cac394-999e-4b1b-93f6-9d413db614c9 admin admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:47:24.082122 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2337f439-ae5a-4c80-860d-169da9a4f682 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbddf971a2354bbdbb9da1d63f3abf88', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0788e6725a84e57a4174284c1845755', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:47:24.083043 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2337f439-ae5a-4c80-860d-169da9a4f682 service cinder] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:47:24.083349 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2337f439-ae5a-4c80-860d-169da9a4f682 service cinder] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:47:24.083622 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2337f439-ae5a-4c80-860d-169da9a4f682 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:47:24.092524 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2337f439-ae5a-4c80-860d-169da9a4f682 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:47:24.093939 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2337f439-ae5a-4c80-860d-169da9a4f682 service cinder] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:47:24.103398 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1313/2610] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 1 16:47:24 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 16:47:24.148003 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf5f1f4-44bf-4ae1-8b67-c701a5407c10 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:47:24.149409 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-fbf5f1f4-44bf-4ae1-8b67-c701a5407c10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:47:24.161440 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf5f1f4-44bf-4ae1-8b67-c701a5407c10 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:47:24.165027 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f4b0e7-fed4-4c78-b929-250168e4ab35 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbddf971a2354bbdbb9da1d63f3abf88', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0788e6725a84e57a4174284c1845755', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:47:24.165654 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f4b0e7-fed4-4c78-b929-250168e4ab35 service cinder] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:47:24.165941 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f4b0e7-fed4-4c78-b929-250168e4ab35 service cinder] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:47:24.166221 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f4b0e7-fed4-4c78-b929-250168e4ab35 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:47:24.223984 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f4b0e7-fed4-4c78-b929-250168e4ab35 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:47:24.225423 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f4b0e7-fed4-4c78-b929-250168e4ab35 service cinder] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:47:24.234487 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1298/2611] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 1 16:47:24 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 16:47:27.468304 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2337f439-ae5a-4c80-860d-169da9a4f682 service cinder] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:47:27.530718 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2337f439-ae5a-4c80-860d-169da9a4f682 service cinder] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:47:27.536423 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb75375-aaed-4a0e-aa79-a223c3554051 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:47:27.537275 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb75375-aaed-4a0e-aa79-a223c3554051 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:47:27.537554 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb75375-aaed-4a0e-aa79-a223c3554051 admin admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:47:27.537825 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb75375-aaed-4a0e-aa79-a223c3554051 admin admin] PATH_INFO: `/v3/users/6031243ff03b493b9d7d9587a5571595` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:47:27.539031 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb75375-aaed-4a0e-aa79-a223c3554051 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6031243ff03b493b9d7d9587a5571595)` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:47:27.540943 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb75375-aaed-4a0e-aa79-a223c3554051 admin admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:47:27.586471 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-5eb75375-aaed-4a0e-aa79-a223c3554051 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6031243ff03b493b9d7d9587a5571595'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:47:27.599156 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-5eb75375-aaed-4a0e-aa79-a223c3554051 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6031243ff03b493b9d7d9587a5571595'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:47:27.605864 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-5eb75375-aaed-4a0e-aa79-a223c3554051 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6031243ff03b493b9d7d9587a5571595'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:47:27.606574 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-5eb75375-aaed-4a0e-aa79-a223c3554051 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6031243ff03b493b9d7d9587a5571595'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:47:27.607181 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-5eb75375-aaed-4a0e-aa79-a223c3554051 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6031243ff03b493b9d7d9587a5571595'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:47:27.619820 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1314/2612] 10.222.0.57 () {58 vars in 1223 bytes} [Fri Jul 1 16:47:27 2022] DELETE /identity/v3/users/6031243ff03b493b9d7d9587a5571595 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 16:47:27.624439 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e676d8db-cf5d-424a-8ebe-38bf3ad9346c None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:47:27.624938 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e676d8db-cf5d-424a-8ebe-38bf3ad9346c None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:47:27.625033 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e676d8db-cf5d-424a-8ebe-38bf3ad9346c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:47:27.642945 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-e676d8db-cf5d-424a-8ebe-38bf3ad9346c None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:47:27.644725 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-e676d8db-cf5d-424a-8ebe-38bf3ad9346c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:47:27.704400 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1299/2613] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 1 16:47:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 16:47:27.715990 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb75375-aaed-4a0e-aa79-a223c3554051 admin admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:47:27.717533 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-5eb75375-aaed-4a0e-aa79-a223c3554051 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:47:27.748266 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:47:27.788312 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb75375-aaed-4a0e-aa79-a223c3554051 admin admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:47:27.792000 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2253f4a-397c-4ce7-ac4c-88901b8f0973 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ed200ce29684c5f80354fa9a8638fe2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0788e6725a84e57a4174284c1845755', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:47:27.792652 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2253f4a-397c-4ce7-ac4c-88901b8f0973 service neutron] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:47:27.792935 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2253f4a-397c-4ce7-ac4c-88901b8f0973 service neutron] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:47:27.793208 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2253f4a-397c-4ce7-ac4c-88901b8f0973 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:47:27.803332 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2253f4a-397c-4ce7-ac4c-88901b8f0973 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:47:27.804754 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2253f4a-397c-4ce7-ac4c-88901b8f0973 service neutron] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:47:27.815380 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1315/2614] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 1 16:47:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 16:47:28.208409 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e676d8db-cf5d-424a-8ebe-38bf3ad9346c None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:47:28.209905 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-e676d8db-cf5d-424a-8ebe-38bf3ad9346c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:47:28.223420 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e676d8db-cf5d-424a-8ebe-38bf3ad9346c None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:47:28.228266 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35834e14-8536-4c1b-ad9b-90ad539fd0e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:47:28.229150 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35834e14-8536-4c1b-ad9b-90ad539fd0e3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:47:28.229434 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35834e14-8536-4c1b-ad9b-90ad539fd0e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:47:28.229713 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35834e14-8536-4c1b-ad9b-90ad539fd0e3 admin admin] PATH_INFO: `/v3/projects/08835d4a4bc049b7b4e6c3476a306a47` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:47:28.230949 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35834e14-8536-4c1b-ad9b-90ad539fd0e3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=08835d4a4bc049b7b4e6c3476a306a47)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:47:28.232887 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35834e14-8536-4c1b-ad9b-90ad539fd0e3 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:47:28.281919 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-35834e14-8536-4c1b-ad9b-90ad539fd0e3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '08835d4a4bc049b7b4e6c3476a306a47', 'request_id': 'req-35834e14-8536-4c1b-ad9b-90ad539fd0e3'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:47:28.287624 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-35834e14-8536-4c1b-ad9b-90ad539fd0e3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '08835d4a4bc049b7b4e6c3476a306a47', 'request_id': 'req-35834e14-8536-4c1b-ad9b-90ad539fd0e3'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:47:28.293813 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-35834e14-8536-4c1b-ad9b-90ad539fd0e3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '08835d4a4bc049b7b4e6c3476a306a47', 'request_id': 'req-35834e14-8536-4c1b-ad9b-90ad539fd0e3'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:47:28.306609 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-35834e14-8536-4c1b-ad9b-90ad539fd0e3 admin admin] The token cache is being invalidate because project 08835d4a4bc049b7b4e6c3476a306a47 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80311) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 01 16:47:28.306894 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-35834e14-8536-4c1b-ad9b-90ad539fd0e3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:47:28.307514 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-35834e14-8536-4c1b-ad9b-90ad539fd0e3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:47:28.309043 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1300/2615] 10.222.0.57 () {58 vars in 1232 bytes} [Fri Jul 1 16:47:28 2022] DELETE /identity/v3/projects/08835d4a4bc049b7b4e6c3476a306a47 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 16:47:28.312984 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2253f4a-397c-4ce7-ac4c-88901b8f0973 service neutron] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:47:28.323045 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2253f4a-397c-4ce7-ac4c-88901b8f0973 service neutron] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:47:28.381231 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe548eb2-ea58-4c69-907d-3f64fe2cdc63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:47:28.382097 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe548eb2-ea58-4c69-907d-3f64fe2cdc63 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:47:28.382403 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe548eb2-ea58-4c69-907d-3f64fe2cdc63 admin admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:47:28.382687 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe548eb2-ea58-4c69-907d-3f64fe2cdc63 admin admin] PATH_INFO: `/v3/users/6183c7d41fa04c90b72382c306640817` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:47:28.383876 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe548eb2-ea58-4c69-907d-3f64fe2cdc63 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6183c7d41fa04c90b72382c306640817)` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:47:28.385834 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe548eb2-ea58-4c69-907d-3f64fe2cdc63 admin admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:47:28.432887 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-fe548eb2-ea58-4c69-907d-3f64fe2cdc63 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6183c7d41fa04c90b72382c306640817'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:47:28.445478 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-fe548eb2-ea58-4c69-907d-3f64fe2cdc63 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6183c7d41fa04c90b72382c306640817'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:47:28.453524 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-fe548eb2-ea58-4c69-907d-3f64fe2cdc63 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6183c7d41fa04c90b72382c306640817'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:47:28.454440 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-fe548eb2-ea58-4c69-907d-3f64fe2cdc63 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6183c7d41fa04c90b72382c306640817'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:47:28.455219 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-fe548eb2-ea58-4c69-907d-3f64fe2cdc63 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6183c7d41fa04c90b72382c306640817'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:47:28.469110 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1316/2616] 10.222.0.57 () {58 vars in 1223 bytes} [Fri Jul 1 16:47:28 2022] DELETE /identity/v3/users/6183c7d41fa04c90b72382c306640817 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 16:47:28.978318 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35834e14-8536-4c1b-ad9b-90ad539fd0e3 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:47:28.982245 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-35834e14-8536-4c1b-ad9b-90ad539fd0e3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:47:28.997865 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35834e14-8536-4c1b-ad9b-90ad539fd0e3 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:47:29.005159 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:47:29.075438 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300e786d-f57e-4c21-9881-057cb05d56a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:47:29.076316 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300e786d-f57e-4c21-9881-057cb05d56a2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:47:29.076603 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300e786d-f57e-4c21-9881-057cb05d56a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:47:29.076880 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300e786d-f57e-4c21-9881-057cb05d56a2 admin admin] PATH_INFO: `/v3/projects/309d7e719187462c87649aaf1ac30679` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:47:29.078472 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300e786d-f57e-4c21-9881-057cb05d56a2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=309d7e719187462c87649aaf1ac30679)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:47:29.081768 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300e786d-f57e-4c21-9881-057cb05d56a2 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:47:29.145959 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-300e786d-f57e-4c21-9881-057cb05d56a2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '309d7e719187462c87649aaf1ac30679', 'request_id': 'req-300e786d-f57e-4c21-9881-057cb05d56a2'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:47:29.159744 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-300e786d-f57e-4c21-9881-057cb05d56a2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '309d7e719187462c87649aaf1ac30679', 'request_id': 'req-300e786d-f57e-4c21-9881-057cb05d56a2'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:47:29.165730 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-300e786d-f57e-4c21-9881-057cb05d56a2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '309d7e719187462c87649aaf1ac30679', 'request_id': 'req-300e786d-f57e-4c21-9881-057cb05d56a2'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:47:29.180312 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-300e786d-f57e-4c21-9881-057cb05d56a2 admin admin] The token cache is being invalidate because project 309d7e719187462c87649aaf1ac30679 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80311) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 01 16:47:29.180733 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-300e786d-f57e-4c21-9881-057cb05d56a2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:47:29.181373 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-300e786d-f57e-4c21-9881-057cb05d56a2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:47:29.182946 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1301/2617] 10.222.0.57 () {58 vars in 1232 bytes} [Fri Jul 1 16:47:28 2022] DELETE /identity/v3/projects/309d7e719187462c87649aaf1ac30679 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 16:47:29.187958 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef157657-9823-4fbc-b415-06b091c993f5 None None] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:47:29.188261 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef157657-9823-4fbc-b415-06b091c993f5 None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:47:29.188543 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef157657-9823-4fbc-b415-06b091c993f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:47:29.208611 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.auth.core [None req-ef157657-9823-4fbc-b415-06b091c993f5 None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80312) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:47:29.209930 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-ef157657-9823-4fbc-b415-06b091c993f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:47:29.255603 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:47:29.279143 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1317/2618] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 1 16:47:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 01 16:47:29.288414 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9b025c-cf2b-43c1-9e82-0b7e29b4c617 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:47:29.288792 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9b025c-cf2b-43c1-9e82-0b7e29b4c617 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:47:29.289124 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9b025c-cf2b-43c1-9e82-0b7e29b4c617 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:47:29.310685 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-bc9b025c-cf2b-43c1-9e82-0b7e29b4c617 None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:47:29.311904 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-bc9b025c-cf2b-43c1-9e82-0b7e29b4c617 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:47:29.321132 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1302/2619] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 1 16:47:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 16:47:29.326102 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef157657-9823-4fbc-b415-06b091c993f5 None None] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:47:29.337202 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef157657-9823-4fbc-b415-06b091c993f5 None None] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:47:29.341802 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4405f5a-cd6b-416d-af51-b5a77c8877da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:47:29.343085 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4405f5a-cd6b-416d-af51-b5a77c8877da admin admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:47:29.343383 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4405f5a-cd6b-416d-af51-b5a77c8877da admin admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:47:29.343661 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4405f5a-cd6b-416d-af51-b5a77c8877da admin admin] PATH_INFO: `/v3/domains` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:47:29.344394 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4405f5a-cd6b-416d-af51-b5a77c8877da admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80312) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:47:29.344703 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4405f5a-cd6b-416d-af51-b5a77c8877da admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:47:29.347457 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4405f5a-cd6b-416d-af51-b5a77c8877da admin admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:47:29.359712 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1318/2620] 10.222.0.57 () {58 vars in 1152 bytes} [Fri Jul 1 16:47:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:47:29.364827 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9b025c-cf2b-43c1-9e82-0b7e29b4c617 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:47:29.374937 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9b025c-cf2b-43c1-9e82-0b7e29b4c617 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:47:29.379980 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a644aa5-40aa-470e-96d4-307179e88122 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:47:29.380640 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a644aa5-40aa-470e-96d4-307179e88122 admin admin] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:47:29.380928 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a644aa5-40aa-470e-96d4-307179e88122 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:47:29.381204 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a644aa5-40aa-470e-96d4-307179e88122 admin admin] PATH_INFO: `/v3/projects` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:47:29.381909 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a644aa5-40aa-470e-96d4-307179e88122 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:47:29.383883 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a644aa5-40aa-470e-96d4-307179e88122 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:47:29.414996 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1303/2621] 10.222.0.57 () {60 vars in 1152 bytes} [Fri Jul 1 16:47:29 2022] POST /identity/v3/projects => generated 399 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 16:47:29.416319 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4405f5a-cd6b-416d-af51-b5a77c8877da admin admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:47:29.428065 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4405f5a-cd6b-416d-af51-b5a77c8877da admin admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:47:29.488529 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84221bf8-3d71-4119-a90d-5a991a058ee7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:47:29.489217 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84221bf8-3d71-4119-a90d-5a991a058ee7 admin admin] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:47:29.489487 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84221bf8-3d71-4119-a90d-5a991a058ee7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:47:29.489768 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84221bf8-3d71-4119-a90d-5a991a058ee7 admin admin] PATH_INFO: `/v3/users` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:47:29.490503 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84221bf8-3d71-4119-a90d-5a991a058ee7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:47:29.492455 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84221bf8-3d71-4119-a90d-5a991a058ee7 admin admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:47:29.515565 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1319/2622] 10.222.0.57 () {60 vars in 1143 bytes} [Fri Jul 1 16:47:29 2022] POST /identity/v3/users => generated 300 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 16:47:29.517039 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a644aa5-40aa-470e-96d4-307179e88122 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:47:29.527892 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a644aa5-40aa-470e-96d4-307179e88122 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:47:29.532547 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58bea254-7ead-42c4-bedc-a505e2ba7d01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:47:29.534208 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bea254-7ead-42c4-bedc-a505e2ba7d01 admin admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:47:29.534527 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bea254-7ead-42c4-bedc-a505e2ba7d01 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:47:29.534811 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bea254-7ead-42c4-bedc-a505e2ba7d01 admin admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:47:29.535544 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58bea254-7ead-42c4-bedc-a505e2ba7d01 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:47:29.537139 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58bea254-7ead-42c4-bedc-a505e2ba7d01 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:47:29.547518 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1304/2623] 10.222.0.57 () {58 vars in 1121 bytes} [Fri Jul 1 16:47:29 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 16:47:29.552239 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84221bf8-3d71-4119-a90d-5a991a058ee7 admin admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:47:29.562480 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84221bf8-3d71-4119-a90d-5a991a058ee7 admin admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:47:29.567525 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc41f20c-32a6-4948-a90c-b00e6decd99b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:47:29.568198 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc41f20c-32a6-4948-a90c-b00e6decd99b admin admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:47:29.568484 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc41f20c-32a6-4948-a90c-b00e6decd99b admin admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:47:29.568764 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc41f20c-32a6-4948-a90c-b00e6decd99b admin admin] PATH_INFO: `/v3/roles` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:47:29.569500 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc41f20c-32a6-4948-a90c-b00e6decd99b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:47:29.571144 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc41f20c-32a6-4948-a90c-b00e6decd99b admin admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:47:29.581582 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1320/2624] 10.222.0.57 () {58 vars in 1121 bytes} [Fri Jul 1 16:47:29 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 16:47:29.586051 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58bea254-7ead-42c4-bedc-a505e2ba7d01 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:47:29.597860 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58bea254-7ead-42c4-bedc-a505e2ba7d01 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:47:29.604498 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366fb8e9-5020-44df-9463-9c7100ff0bd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:47:29.605444 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366fb8e9-5020-44df-9463-9c7100ff0bd1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:47:29.605796 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366fb8e9-5020-44df-9463-9c7100ff0bd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:47:29.606196 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366fb8e9-5020-44df-9463-9c7100ff0bd1 admin admin] PATH_INFO: `/v3/projects/6f5e1be8da4b4b13a18e1d32a53a692b/users/8165c20454cb48e0a3e9e1cab7442647/roles/983be40bdab04c478eaa90c6b6a46b14` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:47:29.637864 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366fb8e9-5020-44df-9463-9c7100ff0bd1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6f5e1be8da4b4b13a18e1d32a53a692b, user_id=8165c20454cb48e0a3e9e1cab7442647, role_id=983be40bdab04c478eaa90c6b6a46b14)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:47:29.643325 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366fb8e9-5020-44df-9463-9c7100ff0bd1 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:47:29.669578 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1305/2625] 10.222.0.57 () {60 vars in 1482 bytes} [Fri Jul 1 16:47:29 2022] PUT /identity/v3/projects/6f5e1be8da4b4b13a18e1d32a53a692b/users/8165c20454cb48e0a3e9e1cab7442647/roles/983be40bdab04c478eaa90c6b6a46b14 => generated 0 bytes in 84 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 16:47:29.680437 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6d4b76-1657-422c-a61a-2fd73151ce2b None None] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:47:29.680728 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6d4b76-1657-422c-a61a-2fd73151ce2b None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:47:29.681009 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6d4b76-1657-422c-a61a-2fd73151ce2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:47:29.721941 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.auth.core [None req-6a6d4b76-1657-422c-a61a-2fd73151ce2b None None] MFA Rules not processed for user `8165c20454cb48e0a3e9e1cab7442647`. Rule list: `[]` (Enabled: `True`). {{(pid=80312) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:47:29.723235 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-6a6d4b76-1657-422c-a61a-2fd73151ce2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:47:29.805610 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1321/2626] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 1 16:47:29 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 16:47:29.807244 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366fb8e9-5020-44df-9463-9c7100ff0bd1 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:47:29.864204 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:47:29.892378 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366fb8e9-5020-44df-9463-9c7100ff0bd1 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:47:29.897333 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5b4f58-8806-46fb-865d-f9ab0789c799 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:47:29.898139 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5b4f58-8806-46fb-865d-f9ab0789c799 admin admin] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:47:29.898457 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5b4f58-8806-46fb-865d-f9ab0789c799 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:47:29.898735 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5b4f58-8806-46fb-865d-f9ab0789c799 admin admin] PATH_INFO: `/v3/projects` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:47:29.899447 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd5b4f58-8806-46fb-865d-f9ab0789c799 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:47:29.901456 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd5b4f58-8806-46fb-865d-f9ab0789c799 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:47:29.931355 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1306/2627] 10.222.0.57 () {60 vars in 1152 bytes} [Fri Jul 1 16:47:29 2022] POST /identity/v3/projects => generated 399 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 16:47:29.935401 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6d4b76-1657-422c-a61a-2fd73151ce2b None None] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:47:29.947282 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6d4b76-1657-422c-a61a-2fd73151ce2b None None] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:47:29.991543 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:47:30.020385 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb24e084-944e-43a0-af01-43a1bff73107 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:47:30.021184 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb24e084-944e-43a0-af01-43a1bff73107 admin admin] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:47:30.021478 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb24e084-944e-43a0-af01-43a1bff73107 admin admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:47:30.021748 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb24e084-944e-43a0-af01-43a1bff73107 admin admin] PATH_INFO: `/v3/users` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:47:30.022480 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb24e084-944e-43a0-af01-43a1bff73107 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:47:30.024425 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb24e084-944e-43a0-af01-43a1bff73107 admin admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:47:30.050137 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1322/2628] 10.222.0.57 () {60 vars in 1143 bytes} [Fri Jul 1 16:47:29 2022] POST /identity/v3/users => generated 306 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 01 16:47:30.054293 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5b4f58-8806-46fb-865d-f9ab0789c799 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:47:30.067193 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5b4f58-8806-46fb-865d-f9ab0789c799 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:47:30.073857 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0793df-703d-4a45-a069-24de699324ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:47:30.075106 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0793df-703d-4a45-a069-24de699324ce admin admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:47:30.075106 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0793df-703d-4a45-a069-24de699324ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:47:30.075495 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0793df-703d-4a45-a069-24de699324ce admin admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:47:30.077051 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0793df-703d-4a45-a069-24de699324ce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:47:30.079109 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0793df-703d-4a45-a069-24de699324ce admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:47:30.092034 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1307/2629] 10.222.0.57 () {58 vars in 1121 bytes} [Fri Jul 1 16:47:30 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 01 16:47:30.096892 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb24e084-944e-43a0-af01-43a1bff73107 admin admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:47:30.107550 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb24e084-944e-43a0-af01-43a1bff73107 admin admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:47:30.113463 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e364b19a-e5e9-4edd-b9d6-fa85a1bde165 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:47:30.114404 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e364b19a-e5e9-4edd-b9d6-fa85a1bde165 admin admin] REQUEST_METHOD: `PUT` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:47:30.114691 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e364b19a-e5e9-4edd-b9d6-fa85a1bde165 admin admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:47:30.114972 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e364b19a-e5e9-4edd-b9d6-fa85a1bde165 admin admin] PATH_INFO: `/v3/projects/0cad3ef9563848c4aa09a40daedd2f8a/users/a9af3de7189d431eb31be820ab4048d9/roles/84ee739146934c1ab32c501ff8093644` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:47:30.135252 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e364b19a-e5e9-4edd-b9d6-fa85a1bde165 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0cad3ef9563848c4aa09a40daedd2f8a, user_id=a9af3de7189d431eb31be820ab4048d9, role_id=84ee739146934c1ab32c501ff8093644)` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:47:30.137940 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e364b19a-e5e9-4edd-b9d6-fa85a1bde165 admin admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:47:30.152990 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1323/2630] 10.222.0.57 () {60 vars in 1482 bytes} [Fri Jul 1 16:47:30 2022] PUT /identity/v3/projects/0cad3ef9563848c4aa09a40daedd2f8a/users/a9af3de7189d431eb31be820ab4048d9/roles/84ee739146934c1ab32c501ff8093644 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 16:47:30.156961 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0793df-703d-4a45-a069-24de699324ce admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:47:30.167633 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0793df-703d-4a45-a069-24de699324ce admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:47:30.239190 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4741e4d1-6f9b-4431-8d39-05c8421598b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:47:30.239872 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4741e4d1-6f9b-4431-8d39-05c8421598b9 admin admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:47:30.240159 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4741e4d1-6f9b-4431-8d39-05c8421598b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:47:30.240441 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4741e4d1-6f9b-4431-8d39-05c8421598b9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:47:30.241179 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4741e4d1-6f9b-4431-8d39-05c8421598b9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:47:30.242820 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4741e4d1-6f9b-4431-8d39-05c8421598b9 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:47:30.259514 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1308/2631] 10.222.0.57 () {58 vars in 1121 bytes} [Fri Jul 1 16:47:30 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 16:47:30.261278 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e364b19a-e5e9-4edd-b9d6-fa85a1bde165 admin admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:47:30.273394 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e364b19a-e5e9-4edd-b9d6-fa85a1bde165 admin admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:47:30.278545 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7c49d5-3bc9-41e8-a7da-660cee29253d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:47:30.279547 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7c49d5-3bc9-41e8-a7da-660cee29253d admin admin] REQUEST_METHOD: `PUT` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:47:30.279952 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7c49d5-3bc9-41e8-a7da-660cee29253d admin admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:47:30.280339 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7c49d5-3bc9-41e8-a7da-660cee29253d admin admin] PATH_INFO: `/v3/projects/0cad3ef9563848c4aa09a40daedd2f8a/users/a9af3de7189d431eb31be820ab4048d9/roles/84ee739146934c1ab32c501ff8093644` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:47:30.282445 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef7c49d5-3bc9-41e8-a7da-660cee29253d admin admin] RBAC: Authorizing `identity:create_grant(project_id=0cad3ef9563848c4aa09a40daedd2f8a, user_id=a9af3de7189d431eb31be820ab4048d9, role_id=84ee739146934c1ab32c501ff8093644)` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:47:30.285266 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef7c49d5-3bc9-41e8-a7da-660cee29253d admin admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:47:30.296577 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1324/2632] 10.222.0.57 () {60 vars in 1482 bytes} [Fri Jul 1 16:47:30 2022] PUT /identity/v3/projects/0cad3ef9563848c4aa09a40daedd2f8a/users/a9af3de7189d431eb31be820ab4048d9/roles/84ee739146934c1ab32c501ff8093644 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 16:47:30.308126 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038ea37e-e63c-41d3-9d36-801b8623834b None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:47:30.308462 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038ea37e-e63c-41d3-9d36-801b8623834b None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:47:30.308814 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038ea37e-e63c-41d3-9d36-801b8623834b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:47:30.337053 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-038ea37e-e63c-41d3-9d36-801b8623834b None None] MFA Rules not processed for user `a9af3de7189d431eb31be820ab4048d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:47:30.338406 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-038ea37e-e63c-41d3-9d36-801b8623834b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:47:30.414327 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1309/2633] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 1 16:47:30 2022] POST /identity/v3/auth/tokens => generated 3333 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 16:47:30.428839 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7c49d5-3bc9-41e8-a7da-660cee29253d admin admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:47:30.431482 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-ef7c49d5-3bc9-41e8-a7da-660cee29253d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:47:30.513249 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7c49d5-3bc9-41e8-a7da-660cee29253d admin admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:47:30.517064 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d153ed51-962e-4c04-a277-f064bd95502d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbddf971a2354bbdbb9da1d63f3abf88', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0788e6725a84e57a4174284c1845755', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:47:30.518660 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d153ed51-962e-4c04-a277-f064bd95502d service cinder] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:47:30.519025 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d153ed51-962e-4c04-a277-f064bd95502d service cinder] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:47:30.519376 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d153ed51-962e-4c04-a277-f064bd95502d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:47:30.531115 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d153ed51-962e-4c04-a277-f064bd95502d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:47:30.532723 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d153ed51-962e-4c04-a277-f064bd95502d service cinder] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:47:30.547816 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1325/2634] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 1 16:47:30 2022] GET /identity/v3/auth/tokens => generated 3333 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 16:47:30.631014 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038ea37e-e63c-41d3-9d36-801b8623834b None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:47:30.641143 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038ea37e-e63c-41d3-9d36-801b8623834b None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:47:30.645086 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0617a19f-7748-43b4-b29f-95cc6c17ecf1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbddf971a2354bbdbb9da1d63f3abf88', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0788e6725a84e57a4174284c1845755', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:47:30.645732 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0617a19f-7748-43b4-b29f-95cc6c17ecf1 service cinder] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:47:30.646015 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0617a19f-7748-43b4-b29f-95cc6c17ecf1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:47:30.646290 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0617a19f-7748-43b4-b29f-95cc6c17ecf1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:47:30.678039 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:47:30.709044 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0617a19f-7748-43b4-b29f-95cc6c17ecf1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:47:30.710587 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0617a19f-7748-43b4-b29f-95cc6c17ecf1 service cinder] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:47:30.720038 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1310/2635] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 1 16:47:30 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 16:47:31.336809 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4acf80-ac4f-4900-b21f-e8de6fea05a0 None None] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:47:31.337106 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4acf80-ac4f-4900-b21f-e8de6fea05a0 None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:47:31.337391 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4acf80-ac4f-4900-b21f-e8de6fea05a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:47:31.355583 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.auth.core [None req-af4acf80-ac4f-4900-b21f-e8de6fea05a0 None None] MFA Rules not processed for user `d6589e2636dd46838a3a0c66d18e0d00`. Rule list: `[]` (Enabled: `True`). {{(pid=80312) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:47:31.356792 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-af4acf80-ac4f-4900-b21f-e8de6fea05a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:47:31.414022 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1326/2636] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 1 16:47:31 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 16:47:31.737006 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0617a19f-7748-43b4-b29f-95cc6c17ecf1 service cinder] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:47:31.738510 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-0617a19f-7748-43b4-b29f-95cc6c17ecf1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:47:31.799082 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0617a19f-7748-43b4-b29f-95cc6c17ecf1 service cinder] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:47:31.802968 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e941e05-b359-463b-9204-3af01b5b43bf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ed200ce29684c5f80354fa9a8638fe2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0788e6725a84e57a4174284c1845755', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:47:31.803766 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e941e05-b359-463b-9204-3af01b5b43bf service neutron] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:47:31.803925 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e941e05-b359-463b-9204-3af01b5b43bf service neutron] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:47:31.804206 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e941e05-b359-463b-9204-3af01b5b43bf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:47:31.806018 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-3e941e05-b359-463b-9204-3af01b5b43bf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:47:31.816512 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e941e05-b359-463b-9204-3af01b5b43bf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:47:31.817944 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e941e05-b359-463b-9204-3af01b5b43bf service neutron] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:47:31.826262 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1311/2637] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 1 16:47:31 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 16:47:34.901447 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4acf80-ac4f-4900-b21f-e8de6fea05a0 None None] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:47:34.948457 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:47:34.979418 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4acf80-ac4f-4900-b21f-e8de6fea05a0 None None] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:47:34.985538 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29061668-ed63-4e00-84a0-9a49775f54dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:47:34.987182 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29061668-ed63-4e00-84a0-9a49775f54dc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:47:34.987639 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29061668-ed63-4e00-84a0-9a49775f54dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:47:34.988097 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29061668-ed63-4e00-84a0-9a49775f54dc admin admin] PATH_INFO: `/v3/users/8165c20454cb48e0a3e9e1cab7442647` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:47:34.989676 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29061668-ed63-4e00-84a0-9a49775f54dc admin admin] RBAC: Authorizing `identity:delete_user(user_id=8165c20454cb48e0a3e9e1cab7442647)` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:47:34.992591 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29061668-ed63-4e00-84a0-9a49775f54dc admin admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:47:35.080634 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-29061668-ed63-4e00-84a0-9a49775f54dc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8165c20454cb48e0a3e9e1cab7442647'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:47:35.100306 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-29061668-ed63-4e00-84a0-9a49775f54dc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8165c20454cb48e0a3e9e1cab7442647'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:47:35.108107 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-29061668-ed63-4e00-84a0-9a49775f54dc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8165c20454cb48e0a3e9e1cab7442647'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:47:35.108860 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-29061668-ed63-4e00-84a0-9a49775f54dc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8165c20454cb48e0a3e9e1cab7442647'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:47:35.109504 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-29061668-ed63-4e00-84a0-9a49775f54dc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8165c20454cb48e0a3e9e1cab7442647'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:47:35.127056 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1327/2638] 10.222.0.57 () {58 vars in 1223 bytes} [Fri Jul 1 16:47:34 2022] DELETE /identity/v3/users/8165c20454cb48e0a3e9e1cab7442647 => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 16:47:35.131737 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9bcca0-7a43-47c4-b672-07f085c92ecc None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:47:35.132224 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9bcca0-7a43-47c4-b672-07f085c92ecc None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:47:35.132345 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9bcca0-7a43-47c4-b672-07f085c92ecc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:47:35.169718 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-4e9bcca0-7a43-47c4-b672-07f085c92ecc None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:47:35.172978 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-4e9bcca0-7a43-47c4-b672-07f085c92ecc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:47:35.292971 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1312/2639] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 1 16:47:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 16:47:35.300104 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29061668-ed63-4e00-84a0-9a49775f54dc admin admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:47:35.301739 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-29061668-ed63-4e00-84a0-9a49775f54dc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:47:35.372104 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29061668-ed63-4e00-84a0-9a49775f54dc admin admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:47:35.376617 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983bcc9e-10d4-4f69-9b30-7248c2b9fed8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ed200ce29684c5f80354fa9a8638fe2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0788e6725a84e57a4174284c1845755', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:47:35.377302 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983bcc9e-10d4-4f69-9b30-7248c2b9fed8 service neutron] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:47:35.377579 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983bcc9e-10d4-4f69-9b30-7248c2b9fed8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:47:35.377870 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983bcc9e-10d4-4f69-9b30-7248c2b9fed8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:47:35.392878 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-983bcc9e-10d4-4f69-9b30-7248c2b9fed8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:47:35.394756 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-983bcc9e-10d4-4f69-9b30-7248c2b9fed8 service neutron] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:47:35.409086 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1328/2640] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 1 16:47:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 16:47:35.730674 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9bcca0-7a43-47c4-b672-07f085c92ecc None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:47:35.731898 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-4e9bcca0-7a43-47c4-b672-07f085c92ecc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:47:35.750849 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9bcca0-7a43-47c4-b672-07f085c92ecc None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:47:35.756414 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a404a195-9873-4d37-a83b-603c8db20a37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:47:35.757774 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a404a195-9873-4d37-a83b-603c8db20a37 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:47:35.757774 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a404a195-9873-4d37-a83b-603c8db20a37 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:47:35.758004 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a404a195-9873-4d37-a83b-603c8db20a37 admin admin] PATH_INFO: `/v3/projects/6f5e1be8da4b4b13a18e1d32a53a692b` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:47:35.761076 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a404a195-9873-4d37-a83b-603c8db20a37 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6f5e1be8da4b4b13a18e1d32a53a692b)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:47:35.763348 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a404a195-9873-4d37-a83b-603c8db20a37 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:47:35.840589 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-a404a195-9873-4d37-a83b-603c8db20a37 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6f5e1be8da4b4b13a18e1d32a53a692b', 'request_id': 'req-a404a195-9873-4d37-a83b-603c8db20a37'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:47:35.857345 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-a404a195-9873-4d37-a83b-603c8db20a37 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6f5e1be8da4b4b13a18e1d32a53a692b', 'request_id': 'req-a404a195-9873-4d37-a83b-603c8db20a37'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:47:35.867986 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-a404a195-9873-4d37-a83b-603c8db20a37 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6f5e1be8da4b4b13a18e1d32a53a692b', 'request_id': 'req-a404a195-9873-4d37-a83b-603c8db20a37'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:47:35.873670 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:47:35.891280 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-a404a195-9873-4d37-a83b-603c8db20a37 admin admin] The token cache is being invalidate because project 6f5e1be8da4b4b13a18e1d32a53a692b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80311) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 01 16:47:35.891593 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-a404a195-9873-4d37-a83b-603c8db20a37 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:47:35.892867 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-a404a195-9873-4d37-a83b-603c8db20a37 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:47:35.895750 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1313/2641] 10.222.0.57 () {58 vars in 1232 bytes} [Fri Jul 1 16:47:35 2022] DELETE /identity/v3/projects/6f5e1be8da4b4b13a18e1d32a53a692b => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 16:47:35.901982 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983bcc9e-10d4-4f69-9b30-7248c2b9fed8 service neutron] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:47:35.930789 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983bcc9e-10d4-4f69-9b30-7248c2b9fed8 service neutron] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:47:36.024460 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41357ef2-2069-41d7-a93e-c0450c04bb87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:47:36.026026 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41357ef2-2069-41d7-a93e-c0450c04bb87 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:47:36.026311 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41357ef2-2069-41d7-a93e-c0450c04bb87 admin admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:47:36.026614 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41357ef2-2069-41d7-a93e-c0450c04bb87 admin admin] PATH_INFO: `/v3/users/a9af3de7189d431eb31be820ab4048d9` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:47:36.028445 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41357ef2-2069-41d7-a93e-c0450c04bb87 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a9af3de7189d431eb31be820ab4048d9)` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:47:36.031418 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41357ef2-2069-41d7-a93e-c0450c04bb87 admin admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:47:36.035055 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:47:36.099725 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-41357ef2-2069-41d7-a93e-c0450c04bb87 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a9af3de7189d431eb31be820ab4048d9'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:47:36.114741 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-41357ef2-2069-41d7-a93e-c0450c04bb87 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a9af3de7189d431eb31be820ab4048d9'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:47:36.122603 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-41357ef2-2069-41d7-a93e-c0450c04bb87 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a9af3de7189d431eb31be820ab4048d9'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:47:36.123934 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-41357ef2-2069-41d7-a93e-c0450c04bb87 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a9af3de7189d431eb31be820ab4048d9'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:47:36.124626 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-41357ef2-2069-41d7-a93e-c0450c04bb87 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a9af3de7189d431eb31be820ab4048d9'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:47:36.143312 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1329/2642] 10.222.0.57 () {58 vars in 1223 bytes} [Fri Jul 1 16:47:35 2022] DELETE /identity/v3/users/a9af3de7189d431eb31be820ab4048d9 => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 16:47:36.500919 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a404a195-9873-4d37-a83b-603c8db20a37 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:47:36.503257 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-a404a195-9873-4d37-a83b-603c8db20a37 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:47:36.520352 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a404a195-9873-4d37-a83b-603c8db20a37 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:47:36.596387 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca30c3f-714c-4d2d-9cd9-a28a8c66affc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:47:36.597202 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca30c3f-714c-4d2d-9cd9-a28a8c66affc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:47:36.597531 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca30c3f-714c-4d2d-9cd9-a28a8c66affc admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:47:36.597816 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca30c3f-714c-4d2d-9cd9-a28a8c66affc admin admin] PATH_INFO: `/v3/projects/0cad3ef9563848c4aa09a40daedd2f8a` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:47:36.599191 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca30c3f-714c-4d2d-9cd9-a28a8c66affc admin admin] RBAC: Authorizing `identity:delete_project(project_id=0cad3ef9563848c4aa09a40daedd2f8a)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:47:36.601335 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca30c3f-714c-4d2d-9cd9-a28a8c66affc admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:47:36.653959 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-5ca30c3f-714c-4d2d-9cd9-a28a8c66affc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0cad3ef9563848c4aa09a40daedd2f8a', 'request_id': 'req-5ca30c3f-714c-4d2d-9cd9-a28a8c66affc'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:47:36.660594 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-5ca30c3f-714c-4d2d-9cd9-a28a8c66affc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0cad3ef9563848c4aa09a40daedd2f8a', 'request_id': 'req-5ca30c3f-714c-4d2d-9cd9-a28a8c66affc'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:47:36.666871 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-5ca30c3f-714c-4d2d-9cd9-a28a8c66affc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0cad3ef9563848c4aa09a40daedd2f8a', 'request_id': 'req-5ca30c3f-714c-4d2d-9cd9-a28a8c66affc'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:47:36.682055 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-5ca30c3f-714c-4d2d-9cd9-a28a8c66affc admin admin] The token cache is being invalidate because project 0cad3ef9563848c4aa09a40daedd2f8a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80311) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 01 16:47:36.682364 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-5ca30c3f-714c-4d2d-9cd9-a28a8c66affc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:47:36.683051 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-5ca30c3f-714c-4d2d-9cd9-a28a8c66affc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:47:36.688364 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1314/2643] 10.222.0.57 () {58 vars in 1232 bytes} [Fri Jul 1 16:47:36 2022] DELETE /identity/v3/projects/0cad3ef9563848c4aa09a40daedd2f8a => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 16:47:36.691827 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd036820-e0f9-49d2-b580-b603be9f432d None None] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:47:36.692352 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd036820-e0f9-49d2-b580-b603be9f432d None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:47:36.692697 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd036820-e0f9-49d2-b580-b603be9f432d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:47:36.717148 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.auth.core [None req-fd036820-e0f9-49d2-b580-b603be9f432d None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80312) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:47:36.718364 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-fd036820-e0f9-49d2-b580-b603be9f432d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:47:36.786451 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1330/2644] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 1 16:47:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 16:47:36.794131 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c269209a-3c2c-4d4a-813a-cbc271109f42 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:47:36.794462 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c269209a-3c2c-4d4a-813a-cbc271109f42 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:47:36.794744 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c269209a-3c2c-4d4a-813a-cbc271109f42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:47:36.814945 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-c269209a-3c2c-4d4a-813a-cbc271109f42 None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:47:36.816251 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-c269209a-3c2c-4d4a-813a-cbc271109f42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:47:36.825048 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1315/2645] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 1 16:47:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 16:47:36.828823 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd036820-e0f9-49d2-b580-b603be9f432d None None] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:47:36.840459 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd036820-e0f9-49d2-b580-b603be9f432d None None] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:47:36.845458 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9400ec03-b841-4852-a6d1-15c7def6b463 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:47:36.847136 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9400ec03-b841-4852-a6d1-15c7def6b463 admin admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:47:36.847136 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9400ec03-b841-4852-a6d1-15c7def6b463 admin admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:47:36.847136 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9400ec03-b841-4852-a6d1-15c7def6b463 admin admin] PATH_INFO: `/v3/domains` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:47:36.848530 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9400ec03-b841-4852-a6d1-15c7def6b463 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80312) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:47:36.848738 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9400ec03-b841-4852-a6d1-15c7def6b463 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:47:36.850391 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9400ec03-b841-4852-a6d1-15c7def6b463 admin admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:47:36.865397 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1331/2646] 10.222.0.57 () {58 vars in 1152 bytes} [Fri Jul 1 16:47:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:47:36.869847 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c269209a-3c2c-4d4a-813a-cbc271109f42 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:47:36.879905 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c269209a-3c2c-4d4a-813a-cbc271109f42 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:47:36.884574 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9272a4-a0ce-458e-b5b2-d748f2d5c2fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:47:36.885226 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9272a4-a0ce-458e-b5b2-d748f2d5c2fe admin admin] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:47:36.885514 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9272a4-a0ce-458e-b5b2-d748f2d5c2fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:47:36.885790 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9272a4-a0ce-458e-b5b2-d748f2d5c2fe admin admin] PATH_INFO: `/v3/projects` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:47:36.886518 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf9272a4-a0ce-458e-b5b2-d748f2d5c2fe admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:47:36.888519 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf9272a4-a0ce-458e-b5b2-d748f2d5c2fe admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:47:36.926679 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1316/2647] 10.222.0.57 () {60 vars in 1152 bytes} [Fri Jul 1 16:47:36 2022] POST /identity/v3/projects => generated 371 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 16:47:36.927978 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9400ec03-b841-4852-a6d1-15c7def6b463 admin admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:47:36.938563 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9400ec03-b841-4852-a6d1-15c7def6b463 admin admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:47:37.015598 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6023084-b2bd-4537-865c-b8c9bf959e77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:47:37.016508 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6023084-b2bd-4537-865c-b8c9bf959e77 admin admin] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:47:37.016802 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6023084-b2bd-4537-865c-b8c9bf959e77 admin admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:47:37.017130 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6023084-b2bd-4537-865c-b8c9bf959e77 admin admin] PATH_INFO: `/v3/users` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:47:37.018060 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6023084-b2bd-4537-865c-b8c9bf959e77 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:47:37.020121 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6023084-b2bd-4537-865c-b8c9bf959e77 admin admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:47:37.032190 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:47:37.048900 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9272a4-a0ce-458e-b5b2-d748f2d5c2fe admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:47:37.049852 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1332/2648] 10.222.0.57 () {60 vars in 1143 bytes} [Fri Jul 1 16:47:36 2022] POST /identity/v3/users => generated 286 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 01 16:47:37.060822 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9272a4-a0ce-458e-b5b2-d748f2d5c2fe admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:47:37.065653 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0180e666-3f84-4233-b79b-27f54c3bc632 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:47:37.066384 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0180e666-3f84-4233-b79b-27f54c3bc632 admin admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:47:37.066683 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0180e666-3f84-4233-b79b-27f54c3bc632 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:47:37.067184 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0180e666-3f84-4233-b79b-27f54c3bc632 admin admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:47:37.068013 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0180e666-3f84-4233-b79b-27f54c3bc632 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:47:37.069760 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0180e666-3f84-4233-b79b-27f54c3bc632 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:47:37.081274 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1317/2649] 10.222.0.57 () {58 vars in 1121 bytes} [Fri Jul 1 16:47:37 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 01 16:47:37.085612 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6023084-b2bd-4537-865c-b8c9bf959e77 admin admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:47:37.096432 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6023084-b2bd-4537-865c-b8c9bf959e77 admin admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:47:37.101332 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d578336e-695c-491e-b662-bbdd5c799233 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:47:37.101992 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d578336e-695c-491e-b662-bbdd5c799233 admin admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:47:37.102278 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d578336e-695c-491e-b662-bbdd5c799233 admin admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:47:37.102605 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d578336e-695c-491e-b662-bbdd5c799233 admin admin] PATH_INFO: `/v3/roles` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:47:37.103335 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d578336e-695c-491e-b662-bbdd5c799233 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:47:37.104920 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d578336e-695c-491e-b662-bbdd5c799233 admin admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:47:37.115570 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1333/2650] 10.222.0.57 () {58 vars in 1121 bytes} [Fri Jul 1 16:47:37 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 16:47:37.119567 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0180e666-3f84-4233-b79b-27f54c3bc632 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:47:37.129609 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0180e666-3f84-4233-b79b-27f54c3bc632 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:47:37.134581 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b36b209-c49c-42a8-a7df-0e7fc16626eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:47:37.135476 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b36b209-c49c-42a8-a7df-0e7fc16626eb admin admin] REQUEST_METHOD: `PUT` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:47:37.135763 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b36b209-c49c-42a8-a7df-0e7fc16626eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:47:37.136041 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b36b209-c49c-42a8-a7df-0e7fc16626eb admin admin] PATH_INFO: `/v3/projects/9e116960838044b697367d17a07b1e95/users/653c52960f5443088a94149b6abf8ea5/roles/983be40bdab04c478eaa90c6b6a46b14` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:47:37.152204 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b36b209-c49c-42a8-a7df-0e7fc16626eb admin admin] RBAC: Authorizing `identity:create_grant(project_id=9e116960838044b697367d17a07b1e95, user_id=653c52960f5443088a94149b6abf8ea5, role_id=983be40bdab04c478eaa90c6b6a46b14)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:47:37.154923 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b36b209-c49c-42a8-a7df-0e7fc16626eb admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:47:37.170507 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1318/2651] 10.222.0.57 () {60 vars in 1482 bytes} [Fri Jul 1 16:47:37 2022] PUT /identity/v3/projects/9e116960838044b697367d17a07b1e95/users/653c52960f5443088a94149b6abf8ea5/roles/983be40bdab04c478eaa90c6b6a46b14 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 16:47:37.181754 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a746cf34-9aa2-4452-8be4-4ae259ca006c None None] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:47:37.182045 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a746cf34-9aa2-4452-8be4-4ae259ca006c None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:47:37.182326 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a746cf34-9aa2-4452-8be4-4ae259ca006c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:47:37.212115 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.auth.core [None req-a746cf34-9aa2-4452-8be4-4ae259ca006c None None] MFA Rules not processed for user `653c52960f5443088a94149b6abf8ea5`. Rule list: `[]` (Enabled: `True`). {{(pid=80312) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:47:37.213422 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-a746cf34-9aa2-4452-8be4-4ae259ca006c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:47:37.282617 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1334/2652] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 1 16:47:37 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 16:47:37.294149 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b36b209-c49c-42a8-a7df-0e7fc16626eb admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:47:37.295938 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-1b36b209-c49c-42a8-a7df-0e7fc16626eb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:47:37.325161 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:47:37.366902 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b36b209-c49c-42a8-a7df-0e7fc16626eb admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:47:37.370930 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4589735-0170-41d6-9b0e-b1ef2929bc40 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbddf971a2354bbdbb9da1d63f3abf88', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0788e6725a84e57a4174284c1845755', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:47:37.371586 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4589735-0170-41d6-9b0e-b1ef2929bc40 service cinder] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:47:37.371875 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4589735-0170-41d6-9b0e-b1ef2929bc40 service cinder] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:47:37.372154 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4589735-0170-41d6-9b0e-b1ef2929bc40 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:47:37.382074 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4589735-0170-41d6-9b0e-b1ef2929bc40 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:47:37.383526 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4589735-0170-41d6-9b0e-b1ef2929bc40 service cinder] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:47:37.393303 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1319/2653] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 1 16:47:37 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 16:47:37.413953 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a746cf34-9aa2-4452-8be4-4ae259ca006c None None] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:47:37.487958 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a746cf34-9aa2-4452-8be4-4ae259ca006c None None] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:47:37.492500 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94639cf7-f9c8-4d08-a477-5295da83ab46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:47:37.493375 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94639cf7-f9c8-4d08-a477-5295da83ab46 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:47:37.493660 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94639cf7-f9c8-4d08-a477-5295da83ab46 admin admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:47:37.493936 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94639cf7-f9c8-4d08-a477-5295da83ab46 admin admin] PATH_INFO: `/v3/users/653c52960f5443088a94149b6abf8ea5` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:47:37.495159 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94639cf7-f9c8-4d08-a477-5295da83ab46 admin admin] RBAC: Authorizing `identity:delete_user(user_id=653c52960f5443088a94149b6abf8ea5)` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:47:37.497114 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94639cf7-f9c8-4d08-a477-5295da83ab46 admin admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:47:37.551603 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-94639cf7-f9c8-4d08-a477-5295da83ab46 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '653c52960f5443088a94149b6abf8ea5'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:47:37.567360 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-94639cf7-f9c8-4d08-a477-5295da83ab46 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '653c52960f5443088a94149b6abf8ea5'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:47:37.576073 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-94639cf7-f9c8-4d08-a477-5295da83ab46 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '653c52960f5443088a94149b6abf8ea5'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:47:37.576922 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-94639cf7-f9c8-4d08-a477-5295da83ab46 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '653c52960f5443088a94149b6abf8ea5'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:47:37.577537 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-94639cf7-f9c8-4d08-a477-5295da83ab46 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '653c52960f5443088a94149b6abf8ea5'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:47:37.581478 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:47:37.599757 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1335/2654] 10.222.0.57 () {58 vars in 1223 bytes} [Fri Jul 1 16:47:37 2022] DELETE /identity/v3/users/653c52960f5443088a94149b6abf8ea5 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 16:47:37.604288 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1aebed4-54dc-418b-87ae-3607d075de59 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:47:37.604797 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1aebed4-54dc-418b-87ae-3607d075de59 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:47:37.604892 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1aebed4-54dc-418b-87ae-3607d075de59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:47:37.622799 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-e1aebed4-54dc-418b-87ae-3607d075de59 None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:47:37.624018 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-e1aebed4-54dc-418b-87ae-3607d075de59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:47:37.690650 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1320/2655] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 1 16:47:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 01 16:47:37.700276 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94639cf7-f9c8-4d08-a477-5295da83ab46 admin admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:47:37.701701 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-94639cf7-f9c8-4d08-a477-5295da83ab46 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:47:37.763871 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94639cf7-f9c8-4d08-a477-5295da83ab46 admin admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:47:37.768113 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a627706-b655-4ef3-a581-f4930eff692d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ed200ce29684c5f80354fa9a8638fe2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0788e6725a84e57a4174284c1845755', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:47:37.768754 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a627706-b655-4ef3-a581-f4930eff692d service neutron] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:47:37.769040 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a627706-b655-4ef3-a581-f4930eff692d service neutron] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:47:37.769321 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a627706-b655-4ef3-a581-f4930eff692d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:47:37.778497 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a627706-b655-4ef3-a581-f4930eff692d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:47:37.779934 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a627706-b655-4ef3-a581-f4930eff692d service neutron] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:47:37.789722 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1336/2656] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 1 16:47:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 16:47:38.111997 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1aebed4-54dc-418b-87ae-3607d075de59 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:47:38.115891 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-e1aebed4-54dc-418b-87ae-3607d075de59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:47:38.145606 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1aebed4-54dc-418b-87ae-3607d075de59 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:47:38.151416 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac40a60-0ec8-4b11-8202-563025d73413 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:47:38.152373 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac40a60-0ec8-4b11-8202-563025d73413 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:47:38.152739 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac40a60-0ec8-4b11-8202-563025d73413 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:47:38.153099 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac40a60-0ec8-4b11-8202-563025d73413 admin admin] PATH_INFO: `/v3/projects/9e116960838044b697367d17a07b1e95` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:47:38.154476 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac40a60-0ec8-4b11-8202-563025d73413 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9e116960838044b697367d17a07b1e95)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:47:38.159174 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac40a60-0ec8-4b11-8202-563025d73413 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:47:38.259442 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-9ac40a60-0ec8-4b11-8202-563025d73413 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9e116960838044b697367d17a07b1e95', 'request_id': 'req-9ac40a60-0ec8-4b11-8202-563025d73413'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:47:38.265426 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-9ac40a60-0ec8-4b11-8202-563025d73413 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9e116960838044b697367d17a07b1e95', 'request_id': 'req-9ac40a60-0ec8-4b11-8202-563025d73413'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:47:38.273472 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-9ac40a60-0ec8-4b11-8202-563025d73413 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9e116960838044b697367d17a07b1e95', 'request_id': 'req-9ac40a60-0ec8-4b11-8202-563025d73413'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:47:38.289196 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-9ac40a60-0ec8-4b11-8202-563025d73413 admin admin] The token cache is being invalidate because project 9e116960838044b697367d17a07b1e95 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80311) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 01 16:47:38.289546 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-9ac40a60-0ec8-4b11-8202-563025d73413 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:47:38.290208 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-9ac40a60-0ec8-4b11-8202-563025d73413 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:47:38.292311 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1321/2657] 10.222.0.57 () {58 vars in 1232 bytes} [Fri Jul 1 16:47:38 2022] DELETE /identity/v3/projects/9e116960838044b697367d17a07b1e95 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 16:47:38.301053 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d43b8d9-4d3b-415c-a731-56a30aed4c23 None None] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:47:38.301579 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d43b8d9-4d3b-415c-a731-56a30aed4c23 None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:47:38.301677 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d43b8d9-4d3b-415c-a731-56a30aed4c23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:47:38.321799 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.auth.core [None req-5d43b8d9-4d3b-415c-a731-56a30aed4c23 None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80312) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:47:38.323068 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-5d43b8d9-4d3b-415c-a731-56a30aed4c23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:47:38.385636 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1337/2658] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 1 16:47:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 01 16:47:38.449865 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829c070d-5174-48d6-b9c9-cfaa484b2954 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:47:38.450182 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829c070d-5174-48d6-b9c9-cfaa484b2954 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:47:38.450517 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829c070d-5174-48d6-b9c9-cfaa484b2954 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:47:38.469029 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-829c070d-5174-48d6-b9c9-cfaa484b2954 None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:47:38.471049 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-829c070d-5174-48d6-b9c9-cfaa484b2954 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:47:38.479262 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1322/2659] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 1 16:47:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 16:47:38.483615 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d43b8d9-4d3b-415c-a731-56a30aed4c23 None None] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:47:38.494728 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d43b8d9-4d3b-415c-a731-56a30aed4c23 None None] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:47:38.500795 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051a2ac1-f7fc-48ac-a55c-f41864cf4f08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:47:38.501607 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051a2ac1-f7fc-48ac-a55c-f41864cf4f08 admin admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:47:38.501940 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051a2ac1-f7fc-48ac-a55c-f41864cf4f08 admin admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:47:38.502303 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051a2ac1-f7fc-48ac-a55c-f41864cf4f08 admin admin] PATH_INFO: `/v3/domains` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:47:38.503113 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-051a2ac1-f7fc-48ac-a55c-f41864cf4f08 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80312) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:47:38.503461 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-051a2ac1-f7fc-48ac-a55c-f41864cf4f08 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:47:38.505202 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-051a2ac1-f7fc-48ac-a55c-f41864cf4f08 admin admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:47:38.517462 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1338/2660] 10.222.0.57 () {58 vars in 1152 bytes} [Fri Jul 1 16:47:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:47:38.521994 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829c070d-5174-48d6-b9c9-cfaa484b2954 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:47:38.531637 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829c070d-5174-48d6-b9c9-cfaa484b2954 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:47:38.538148 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2493a357-6094-4ace-a736-c17f761fad55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:47:38.538787 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2493a357-6094-4ace-a736-c17f761fad55 admin admin] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:47:38.539127 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2493a357-6094-4ace-a736-c17f761fad55 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:47:38.539416 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2493a357-6094-4ace-a736-c17f761fad55 admin admin] PATH_INFO: `/v3/projects` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:47:38.540105 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2493a357-6094-4ace-a736-c17f761fad55 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:47:38.542033 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2493a357-6094-4ace-a736-c17f761fad55 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:47:38.571840 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1323/2661] 10.222.0.57 () {60 vars in 1152 bytes} [Fri Jul 1 16:47:38 2022] POST /identity/v3/projects => generated 375 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 16:47:38.573003 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051a2ac1-f7fc-48ac-a55c-f41864cf4f08 admin admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:47:38.583718 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051a2ac1-f7fc-48ac-a55c-f41864cf4f08 admin admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:47:38.643937 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668aedcb-ba01-449b-be57-1d89bec2f4b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:47:38.644605 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668aedcb-ba01-449b-be57-1d89bec2f4b4 admin admin] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:47:38.644888 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668aedcb-ba01-449b-be57-1d89bec2f4b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:47:38.645168 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668aedcb-ba01-449b-be57-1d89bec2f4b4 admin admin] PATH_INFO: `/v3/users` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:47:38.645862 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-668aedcb-ba01-449b-be57-1d89bec2f4b4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:47:38.647913 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-668aedcb-ba01-449b-be57-1d89bec2f4b4 admin admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:47:38.671825 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1339/2662] 10.222.0.57 () {60 vars in 1143 bytes} [Fri Jul 1 16:47:38 2022] POST /identity/v3/users => generated 288 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 16:47:38.672972 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2493a357-6094-4ace-a736-c17f761fad55 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:47:38.685626 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2493a357-6094-4ace-a736-c17f761fad55 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:47:38.690779 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a2f345-0d8e-4bc5-aaaf-1b411c0c2e9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:47:38.691589 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a2f345-0d8e-4bc5-aaaf-1b411c0c2e9b admin admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:47:38.691872 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a2f345-0d8e-4bc5-aaaf-1b411c0c2e9b admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:47:38.692146 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a2f345-0d8e-4bc5-aaaf-1b411c0c2e9b admin admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:47:38.692887 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a2f345-0d8e-4bc5-aaaf-1b411c0c2e9b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:47:38.694624 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a2f345-0d8e-4bc5-aaaf-1b411c0c2e9b admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:47:38.712555 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1324/2663] 10.222.0.57 () {58 vars in 1121 bytes} [Fri Jul 1 16:47:38 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 16:47:38.713993 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668aedcb-ba01-449b-be57-1d89bec2f4b4 admin admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:47:38.728113 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668aedcb-ba01-449b-be57-1d89bec2f4b4 admin admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:47:38.733532 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2202274-effd-40dc-98fe-30809b406415 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:47:38.734236 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2202274-effd-40dc-98fe-30809b406415 admin admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:47:38.734556 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2202274-effd-40dc-98fe-30809b406415 admin admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:47:38.734848 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2202274-effd-40dc-98fe-30809b406415 admin admin] PATH_INFO: `/v3/roles` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:47:38.736254 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2202274-effd-40dc-98fe-30809b406415 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:47:38.737880 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2202274-effd-40dc-98fe-30809b406415 admin admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:47:38.752530 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1340/2664] 10.222.0.57 () {58 vars in 1121 bytes} [Fri Jul 1 16:47:38 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 16:47:38.753685 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a2f345-0d8e-4bc5-aaaf-1b411c0c2e9b admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:47:38.764944 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a2f345-0d8e-4bc5-aaaf-1b411c0c2e9b admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:47:38.769787 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90a2c19-4253-45e4-b708-d4b535289b2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:47:38.770707 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90a2c19-4253-45e4-b708-d4b535289b2c admin admin] REQUEST_METHOD: `PUT` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:47:38.770991 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90a2c19-4253-45e4-b708-d4b535289b2c admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:47:38.771269 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90a2c19-4253-45e4-b708-d4b535289b2c admin admin] PATH_INFO: `/v3/projects/c36ada34f1224eb9981865391806b46e/users/ee4cb5ba84f5460abdc704b31e5bc516/roles/983be40bdab04c478eaa90c6b6a46b14` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:47:38.775149 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:47:38.790160 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c90a2c19-4253-45e4-b708-d4b535289b2c admin admin] RBAC: Authorizing `identity:create_grant(project_id=c36ada34f1224eb9981865391806b46e, user_id=ee4cb5ba84f5460abdc704b31e5bc516, role_id=983be40bdab04c478eaa90c6b6a46b14)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:47:38.792781 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c90a2c19-4253-45e4-b708-d4b535289b2c admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:47:38.816403 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1325/2665] 10.222.0.57 () {60 vars in 1482 bytes} [Fri Jul 1 16:47:38 2022] PUT /identity/v3/projects/c36ada34f1224eb9981865391806b46e/users/ee4cb5ba84f5460abdc704b31e5bc516/roles/983be40bdab04c478eaa90c6b6a46b14 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 16:47:38.818747 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c015ab-d142-4caf-95e4-177ec9db462d None None] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:47:38.819256 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c015ab-d142-4caf-95e4-177ec9db462d None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:47:38.819647 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c015ab-d142-4caf-95e4-177ec9db462d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:47:38.827535 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:DELETE=1,keystone:SELECT=96,keystone:INSERT=3 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:47:38.860422 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.auth.core [None req-52c015ab-d142-4caf-95e4-177ec9db462d None None] MFA Rules not processed for user `ee4cb5ba84f5460abdc704b31e5bc516`. Rule list: `[]` (Enabled: `True`). {{(pid=80312) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:47:38.862269 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-52c015ab-d142-4caf-95e4-177ec9db462d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:47:38.928939 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1341/2666] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 1 16:47:38 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 16:47:38.933719 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90a2c19-4253-45e4-b708-d4b535289b2c admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:47:38.996361 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90a2c19-4253-45e4-b708-d4b535289b2c admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:47:39.001010 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9b210d-3f4d-4396-913b-02c4c6dabbe8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:47:39.002431 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9b210d-3f4d-4396-913b-02c4c6dabbe8 admin admin] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:47:39.002928 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9b210d-3f4d-4396-913b-02c4c6dabbe8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:47:39.003425 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9b210d-3f4d-4396-913b-02c4c6dabbe8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:47:39.004295 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b9b210d-3f4d-4396-913b-02c4c6dabbe8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:47:39.006540 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b9b210d-3f4d-4396-913b-02c4c6dabbe8 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:47:39.031877 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1326/2667] 10.222.0.57 () {60 vars in 1152 bytes} [Fri Jul 1 16:47:38 2022] POST /identity/v3/projects => generated 375 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 16:47:39.036299 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c015ab-d142-4caf-95e4-177ec9db462d None None] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:47:39.047258 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c015ab-d142-4caf-95e4-177ec9db462d None None] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:47:39.103833 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f09459-c409-4cc9-96bc-4699edd21e2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:47:39.104502 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f09459-c409-4cc9-96bc-4699edd21e2e admin admin] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:47:39.104784 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f09459-c409-4cc9-96bc-4699edd21e2e admin admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:47:39.105056 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f09459-c409-4cc9-96bc-4699edd21e2e admin admin] PATH_INFO: `/v3/users` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:47:39.105754 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f09459-c409-4cc9-96bc-4699edd21e2e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:47:39.107694 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f09459-c409-4cc9-96bc-4699edd21e2e admin admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:47:39.128471 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1342/2668] 10.222.0.57 () {60 vars in 1143 bytes} [Fri Jul 1 16:47:39 2022] POST /identity/v3/users => generated 294 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 16:47:39.132462 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9b210d-3f4d-4396-913b-02c4c6dabbe8 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:47:39.143750 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9b210d-3f4d-4396-913b-02c4c6dabbe8 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:47:39.151817 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1582452-83d8-494e-a1c7-9cb3aeb6ed33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:47:39.152483 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1582452-83d8-494e-a1c7-9cb3aeb6ed33 admin admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:47:39.152764 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1582452-83d8-494e-a1c7-9cb3aeb6ed33 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:47:39.153041 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1582452-83d8-494e-a1c7-9cb3aeb6ed33 admin admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:47:39.153775 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1582452-83d8-494e-a1c7-9cb3aeb6ed33 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:47:39.155405 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1582452-83d8-494e-a1c7-9cb3aeb6ed33 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:47:39.167203 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1327/2669] 10.222.0.57 () {58 vars in 1121 bytes} [Fri Jul 1 16:47:39 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 16:47:39.171481 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f09459-c409-4cc9-96bc-4699edd21e2e admin admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:47:39.181588 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f09459-c409-4cc9-96bc-4699edd21e2e admin admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:47:39.186270 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5752a413-4b3c-4a20-8394-5a30a1f03cdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:47:39.187177 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5752a413-4b3c-4a20-8394-5a30a1f03cdb admin admin] REQUEST_METHOD: `PUT` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:47:39.187463 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5752a413-4b3c-4a20-8394-5a30a1f03cdb admin admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:47:39.187746 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5752a413-4b3c-4a20-8394-5a30a1f03cdb admin admin] PATH_INFO: `/v3/projects/9712776a5798444f8f0c9a7e904378f7/users/827b1ca9b8754f409327c366ea58f33f/roles/84ee739146934c1ab32c501ff8093644` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:47:39.203916 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5752a413-4b3c-4a20-8394-5a30a1f03cdb admin admin] RBAC: Authorizing `identity:create_grant(project_id=9712776a5798444f8f0c9a7e904378f7, user_id=827b1ca9b8754f409327c366ea58f33f, role_id=84ee739146934c1ab32c501ff8093644)` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:47:39.207296 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5752a413-4b3c-4a20-8394-5a30a1f03cdb admin admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:47:39.219964 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1343/2670] 10.222.0.57 () {60 vars in 1482 bytes} [Fri Jul 1 16:47:39 2022] PUT /identity/v3/projects/9712776a5798444f8f0c9a7e904378f7/users/827b1ca9b8754f409327c366ea58f33f/roles/84ee739146934c1ab32c501ff8093644 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 16:47:39.224680 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1582452-83d8-494e-a1c7-9cb3aeb6ed33 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:47:39.234914 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1582452-83d8-494e-a1c7-9cb3aeb6ed33 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:47:39.292701 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43613d8c-ea72-4d69-9437-0b52ec198289 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:47:39.293365 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43613d8c-ea72-4d69-9437-0b52ec198289 admin admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:47:39.293649 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43613d8c-ea72-4d69-9437-0b52ec198289 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:47:39.293926 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43613d8c-ea72-4d69-9437-0b52ec198289 admin admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:47:39.294689 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43613d8c-ea72-4d69-9437-0b52ec198289 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:47:39.296274 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43613d8c-ea72-4d69-9437-0b52ec198289 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:47:39.309912 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1328/2671] 10.222.0.57 () {58 vars in 1121 bytes} [Fri Jul 1 16:47:39 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 16:47:39.310936 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5752a413-4b3c-4a20-8394-5a30a1f03cdb admin admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:47:39.321879 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5752a413-4b3c-4a20-8394-5a30a1f03cdb admin admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:47:39.327788 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878f396f-1582-44dc-acfb-a9bc6fe45de4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:47:39.328682 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878f396f-1582-44dc-acfb-a9bc6fe45de4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:47:39.329359 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878f396f-1582-44dc-acfb-a9bc6fe45de4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:47:39.329359 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878f396f-1582-44dc-acfb-a9bc6fe45de4 admin admin] PATH_INFO: `/v3/projects/9712776a5798444f8f0c9a7e904378f7/users/827b1ca9b8754f409327c366ea58f33f/roles/84ee739146934c1ab32c501ff8093644` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:47:39.333303 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-878f396f-1582-44dc-acfb-a9bc6fe45de4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9712776a5798444f8f0c9a7e904378f7, user_id=827b1ca9b8754f409327c366ea58f33f, role_id=84ee739146934c1ab32c501ff8093644)` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:47:39.337122 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-878f396f-1582-44dc-acfb-a9bc6fe45de4 admin admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:47:39.350152 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1344/2672] 10.222.0.57 () {60 vars in 1482 bytes} [Fri Jul 1 16:47:39 2022] PUT /identity/v3/projects/9712776a5798444f8f0c9a7e904378f7/users/827b1ca9b8754f409327c366ea58f33f/roles/84ee739146934c1ab32c501ff8093644 => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 16:47:39.361704 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cffd050-aaed-441e-aee1-a58af6512dad None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:47:39.362478 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cffd050-aaed-441e-aee1-a58af6512dad None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:47:39.362478 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cffd050-aaed-441e-aee1-a58af6512dad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:47:39.393436 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-3cffd050-aaed-441e-aee1-a58af6512dad None None] MFA Rules not processed for user `827b1ca9b8754f409327c366ea58f33f`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:47:39.394789 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-3cffd050-aaed-441e-aee1-a58af6512dad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:47:39.452090 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:47:39.485086 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1329/2673] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 1 16:47:39 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 16:47:39.498280 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878f396f-1582-44dc-acfb-a9bc6fe45de4 admin admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:47:39.500046 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-878f396f-1582-44dc-acfb-a9bc6fe45de4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:47:39.551827 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:47:39.564196 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cffd050-aaed-441e-aee1-a58af6512dad None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:47:39.564778 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878f396f-1582-44dc-acfb-a9bc6fe45de4 admin admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:47:39.565712 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-3cffd050-aaed-441e-aee1-a58af6512dad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:47:39.568444 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bbef51a-28f6-4e88-a8e0-3bed4208407a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbddf971a2354bbdbb9da1d63f3abf88', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0788e6725a84e57a4174284c1845755', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:47:39.569098 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbef51a-28f6-4e88-a8e0-3bed4208407a service cinder] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:47:39.569386 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbef51a-28f6-4e88-a8e0-3bed4208407a service cinder] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:47:39.569671 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbef51a-28f6-4e88-a8e0-3bed4208407a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:47:39.625292 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cffd050-aaed-441e-aee1-a58af6512dad None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:47:39.629222 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44137dce-d2b3-41af-82b2-eca937dd0ee9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ed200ce29684c5f80354fa9a8638fe2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0788e6725a84e57a4174284c1845755', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:47:39.629908 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44137dce-d2b3-41af-82b2-eca937dd0ee9 service neutron] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:47:39.630192 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44137dce-d2b3-41af-82b2-eca937dd0ee9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:47:39.630503 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44137dce-d2b3-41af-82b2-eca937dd0ee9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:47:39.632447 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-44137dce-d2b3-41af-82b2-eca937dd0ee9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:47:39.635250 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bbef51a-28f6-4e88-a8e0-3bed4208407a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:47:39.636831 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bbef51a-28f6-4e88-a8e0-3bed4208407a service cinder] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:47:39.647496 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1345/2674] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 1 16:47:39 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 16:47:39.702062 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44137dce-d2b3-41af-82b2-eca937dd0ee9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:47:39.703698 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44137dce-d2b3-41af-82b2-eca937dd0ee9 service neutron] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:47:39.714665 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1330/2675] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 1 16:47:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 01 16:47:42.206004 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bbef51a-28f6-4e88-a8e0-3bed4208407a service cinder] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:47:42.221611 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bbef51a-28f6-4e88-a8e0-3bed4208407a service cinder] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:47:42.226939 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579e8fe1-0023-46e8-aa95-55905f24337b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:47:42.227867 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579e8fe1-0023-46e8-aa95-55905f24337b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:47:42.228151 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579e8fe1-0023-46e8-aa95-55905f24337b admin admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:47:42.228424 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579e8fe1-0023-46e8-aa95-55905f24337b admin admin] PATH_INFO: `/v3/users/ee4cb5ba84f5460abdc704b31e5bc516` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:47:42.229645 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-579e8fe1-0023-46e8-aa95-55905f24337b admin admin] RBAC: Authorizing `identity:delete_user(user_id=ee4cb5ba84f5460abdc704b31e5bc516)` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:47:42.231755 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-579e8fe1-0023-46e8-aa95-55905f24337b admin admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:47:42.304849 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-579e8fe1-0023-46e8-aa95-55905f24337b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ee4cb5ba84f5460abdc704b31e5bc516'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:47:42.321361 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-579e8fe1-0023-46e8-aa95-55905f24337b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ee4cb5ba84f5460abdc704b31e5bc516'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:47:42.332564 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-579e8fe1-0023-46e8-aa95-55905f24337b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ee4cb5ba84f5460abdc704b31e5bc516'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:47:42.333236 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-579e8fe1-0023-46e8-aa95-55905f24337b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ee4cb5ba84f5460abdc704b31e5bc516'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:47:42.334079 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-579e8fe1-0023-46e8-aa95-55905f24337b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ee4cb5ba84f5460abdc704b31e5bc516'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:47:42.355559 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1346/2676] 10.222.0.57 () {58 vars in 1223 bytes} [Fri Jul 1 16:47:42 2022] DELETE /identity/v3/users/ee4cb5ba84f5460abdc704b31e5bc516 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 16:47:42.357327 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d1e093-2169-4fa2-b9d5-b63edb6e276e None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:47:42.358004 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d1e093-2169-4fa2-b9d5-b63edb6e276e None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:47:42.358172 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d1e093-2169-4fa2-b9d5-b63edb6e276e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:47:42.384258 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-c8d1e093-2169-4fa2-b9d5-b63edb6e276e None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:47:42.385568 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-c8d1e093-2169-4fa2-b9d5-b63edb6e276e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:47:42.478608 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1331/2677] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 1 16:47:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 01 16:47:42.500693 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579e8fe1-0023-46e8-aa95-55905f24337b admin admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:47:42.503014 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-579e8fe1-0023-46e8-aa95-55905f24337b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:47:42.589043 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579e8fe1-0023-46e8-aa95-55905f24337b admin admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:47:42.593118 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ed4fec-50ea-4576-83c2-3eeae14271d7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ed200ce29684c5f80354fa9a8638fe2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0788e6725a84e57a4174284c1845755', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:47:42.593931 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ed4fec-50ea-4576-83c2-3eeae14271d7 service neutron] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:47:42.594320 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ed4fec-50ea-4576-83c2-3eeae14271d7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:47:42.594733 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ed4fec-50ea-4576-83c2-3eeae14271d7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:47:42.606147 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ed4fec-50ea-4576-83c2-3eeae14271d7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:47:42.608079 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ed4fec-50ea-4576-83c2-3eeae14271d7 service neutron] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:47:42.625333 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1347/2678] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 1 16:47:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 16:47:43.003164 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d1e093-2169-4fa2-b9d5-b63edb6e276e None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:47:43.004888 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-c8d1e093-2169-4fa2-b9d5-b63edb6e276e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:47:43.019302 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d1e093-2169-4fa2-b9d5-b63edb6e276e None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:47:43.024344 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551ad494-d3d8-422b-97bb-d353cbe9de9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:47:43.025312 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551ad494-d3d8-422b-97bb-d353cbe9de9b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:47:43.025645 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551ad494-d3d8-422b-97bb-d353cbe9de9b admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:47:43.026060 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551ad494-d3d8-422b-97bb-d353cbe9de9b admin admin] PATH_INFO: `/v3/projects/c36ada34f1224eb9981865391806b46e` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:47:43.027698 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551ad494-d3d8-422b-97bb-d353cbe9de9b admin admin] RBAC: Authorizing `identity:delete_project(project_id=c36ada34f1224eb9981865391806b46e)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:47:43.029462 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551ad494-d3d8-422b-97bb-d353cbe9de9b admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:47:43.067654 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:47:43.089777 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-551ad494-d3d8-422b-97bb-d353cbe9de9b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c36ada34f1224eb9981865391806b46e', 'request_id': 'req-551ad494-d3d8-422b-97bb-d353cbe9de9b'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:47:43.097986 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-551ad494-d3d8-422b-97bb-d353cbe9de9b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c36ada34f1224eb9981865391806b46e', 'request_id': 'req-551ad494-d3d8-422b-97bb-d353cbe9de9b'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:47:43.104813 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-551ad494-d3d8-422b-97bb-d353cbe9de9b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c36ada34f1224eb9981865391806b46e', 'request_id': 'req-551ad494-d3d8-422b-97bb-d353cbe9de9b'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:47:43.113992 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ed4fec-50ea-4576-83c2-3eeae14271d7 service neutron] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:47:43.115559 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-67ed4fec-50ea-4576-83c2-3eeae14271d7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:47:43.121921 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-551ad494-d3d8-422b-97bb-d353cbe9de9b admin admin] The token cache is being invalidate because project c36ada34f1224eb9981865391806b46e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80311) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 01 16:47:43.122216 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-551ad494-d3d8-422b-97bb-d353cbe9de9b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:47:43.122898 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-551ad494-d3d8-422b-97bb-d353cbe9de9b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:47:43.124833 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1332/2679] 10.222.0.57 () {58 vars in 1232 bytes} [Fri Jul 1 16:47:43 2022] DELETE /identity/v3/projects/c36ada34f1224eb9981865391806b46e => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 16:47:43.128593 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551ad494-d3d8-422b-97bb-d353cbe9de9b admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:47:43.129955 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-551ad494-d3d8-422b-97bb-d353cbe9de9b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:47:43.136422 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ed4fec-50ea-4576-83c2-3eeae14271d7 service neutron] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:47:43.147405 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:47:43.206905 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00ce588-4f34-43d4-b685-be7c01818722 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:47:43.207839 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551ad494-d3d8-422b-97bb-d353cbe9de9b admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:47:43.208074 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00ce588-4f34-43d4-b685-be7c01818722 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:47:43.208355 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00ce588-4f34-43d4-b685-be7c01818722 admin admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:47:43.208712 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00ce588-4f34-43d4-b685-be7c01818722 admin admin] PATH_INFO: `/v3/users/e471b31763074167ba81c2c25f39b857` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:47:43.209920 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e00ce588-4f34-43d4-b685-be7c01818722 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e471b31763074167ba81c2c25f39b857)` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:47:43.212126 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e00ce588-4f34-43d4-b685-be7c01818722 admin admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:47:43.212877 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ad3476-a74f-49df-9672-39e87ab64988 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:47:43.213787 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ad3476-a74f-49df-9672-39e87ab64988 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:47:43.214037 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ad3476-a74f-49df-9672-39e87ab64988 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:47:43.214328 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ad3476-a74f-49df-9672-39e87ab64988 admin admin] PATH_INFO: `/v3/users/827b1ca9b8754f409327c366ea58f33f` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:47:43.215635 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ad3476-a74f-49df-9672-39e87ab64988 admin admin] RBAC: Authorizing `identity:delete_user(user_id=827b1ca9b8754f409327c366ea58f33f)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:47:43.217544 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ad3476-a74f-49df-9672-39e87ab64988 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:47:43.269634 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-e00ce588-4f34-43d4-b685-be7c01818722 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e471b31763074167ba81c2c25f39b857'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:47:43.280478 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-d2ad3476-a74f-49df-9672-39e87ab64988 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '827b1ca9b8754f409327c366ea58f33f'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:47:43.281239 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-d2ad3476-a74f-49df-9672-39e87ab64988 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '827b1ca9b8754f409327c366ea58f33f'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:47:43.282705 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-d2ad3476-a74f-49df-9672-39e87ab64988 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '827b1ca9b8754f409327c366ea58f33f'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:47:43.292302 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-d2ad3476-a74f-49df-9672-39e87ab64988 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '827b1ca9b8754f409327c366ea58f33f'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:47:43.292520 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-e00ce588-4f34-43d4-b685-be7c01818722 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e471b31763074167ba81c2c25f39b857'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:47:43.300793 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-e00ce588-4f34-43d4-b685-be7c01818722 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e471b31763074167ba81c2c25f39b857'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:47:43.301559 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-e00ce588-4f34-43d4-b685-be7c01818722 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e471b31763074167ba81c2c25f39b857'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:47:43.302418 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-e00ce588-4f34-43d4-b685-be7c01818722 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e471b31763074167ba81c2c25f39b857'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:47:43.309562 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-d2ad3476-a74f-49df-9672-39e87ab64988 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '827b1ca9b8754f409327c366ea58f33f'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:47:43.320534 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1348/2680] 10.222.0.57 () {58 vars in 1223 bytes} [Fri Jul 1 16:47:43 2022] DELETE /identity/v3/users/e471b31763074167ba81c2c25f39b857 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 16:47:43.322724 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1333/2681] 10.222.0.57 () {58 vars in 1223 bytes} [Fri Jul 1 16:47:43 2022] DELETE /identity/v3/users/827b1ca9b8754f409327c366ea58f33f => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 16:47:43.521757 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00ce588-4f34-43d4-b685-be7c01818722 admin admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:47:43.523221 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-e00ce588-4f34-43d4-b685-be7c01818722 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:47:43.618081 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00ce588-4f34-43d4-b685-be7c01818722 admin admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:47:43.666385 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:47:43.699967 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56bdafb-99a2-4be9-89b0-58d6e81a9920 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:47:43.700787 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ad3476-a74f-49df-9672-39e87ab64988 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:47:43.701278 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56bdafb-99a2-4be9-89b0-58d6e81a9920 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:47:43.701434 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56bdafb-99a2-4be9-89b0-58d6e81a9920 admin admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:47:43.701722 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56bdafb-99a2-4be9-89b0-58d6e81a9920 admin admin] PATH_INFO: `/v3/projects/315dd9cfab134d008039e1229e42a96e` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:47:43.702793 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-d2ad3476-a74f-49df-9672-39e87ab64988 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:47:43.703706 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b56bdafb-99a2-4be9-89b0-58d6e81a9920 admin admin] RBAC: Authorizing `identity:delete_project(project_id=315dd9cfab134d008039e1229e42a96e)` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:47:43.707909 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b56bdafb-99a2-4be9-89b0-58d6e81a9920 admin admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:47:43.720488 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ad3476-a74f-49df-9672-39e87ab64988 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:47:43.725852 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70eb78ad-3ca4-4c93-969f-02b882022c9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:47:43.727128 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70eb78ad-3ca4-4c93-969f-02b882022c9d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:47:43.727491 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70eb78ad-3ca4-4c93-969f-02b882022c9d admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:47:43.727835 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70eb78ad-3ca4-4c93-969f-02b882022c9d admin admin] PATH_INFO: `/v3/projects/9712776a5798444f8f0c9a7e904378f7` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:47:43.729508 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70eb78ad-3ca4-4c93-969f-02b882022c9d admin admin] RBAC: Authorizing `identity:delete_project(project_id=9712776a5798444f8f0c9a7e904378f7)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:47:43.731488 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70eb78ad-3ca4-4c93-969f-02b882022c9d admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:47:43.776252 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:DELETE=18,keystone:SELECT=80,keystone:INSERT=2 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:47:43.790003 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-b56bdafb-99a2-4be9-89b0-58d6e81a9920 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '315dd9cfab134d008039e1229e42a96e', 'request_id': 'req-b56bdafb-99a2-4be9-89b0-58d6e81a9920'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:47:43.796059 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-70eb78ad-3ca4-4c93-969f-02b882022c9d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9712776a5798444f8f0c9a7e904378f7', 'request_id': 'req-70eb78ad-3ca4-4c93-969f-02b882022c9d'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:47:43.799609 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-b56bdafb-99a2-4be9-89b0-58d6e81a9920 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '315dd9cfab134d008039e1229e42a96e', 'request_id': 'req-b56bdafb-99a2-4be9-89b0-58d6e81a9920'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:47:43.803138 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-70eb78ad-3ca4-4c93-969f-02b882022c9d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9712776a5798444f8f0c9a7e904378f7', 'request_id': 'req-70eb78ad-3ca4-4c93-969f-02b882022c9d'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:47:43.806812 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-b56bdafb-99a2-4be9-89b0-58d6e81a9920 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '315dd9cfab134d008039e1229e42a96e', 'request_id': 'req-b56bdafb-99a2-4be9-89b0-58d6e81a9920'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:47:43.812745 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-70eb78ad-3ca4-4c93-969f-02b882022c9d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9712776a5798444f8f0c9a7e904378f7', 'request_id': 'req-70eb78ad-3ca4-4c93-969f-02b882022c9d'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:47:43.821087 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-b56bdafb-99a2-4be9-89b0-58d6e81a9920 admin admin] The token cache is being invalidate because project 315dd9cfab134d008039e1229e42a96e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80312) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 01 16:47:43.821386 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-b56bdafb-99a2-4be9-89b0-58d6e81a9920 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:47:43.822089 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-b56bdafb-99a2-4be9-89b0-58d6e81a9920 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:47:43.823819 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1349/2682] 10.222.0.57 () {58 vars in 1232 bytes} [Fri Jul 1 16:47:43 2022] DELETE /identity/v3/projects/315dd9cfab134d008039e1229e42a96e => generated 0 bytes in 303 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 16:47:43.828722 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c1076d-b5cf-420b-a2e6-9b71b11a0bfc None None] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:47:43.829042 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c1076d-b5cf-420b-a2e6-9b71b11a0bfc None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:47:43.829962 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c1076d-b5cf-420b-a2e6-9b71b11a0bfc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:47:43.830877 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-70eb78ad-3ca4-4c93-969f-02b882022c9d admin admin] The token cache is being invalidate because project 9712776a5798444f8f0c9a7e904378f7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80311) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 01 16:47:43.831113 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-70eb78ad-3ca4-4c93-969f-02b882022c9d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:47:43.832150 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-70eb78ad-3ca4-4c93-969f-02b882022c9d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:47:43.837313 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1334/2683] 10.222.0.57 () {58 vars in 1232 bytes} [Fri Jul 1 16:47:43 2022] DELETE /identity/v3/projects/9712776a5798444f8f0c9a7e904378f7 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 16:47:43.840739 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e2e8a1-bf65-476a-a4d0-3a820ff58162 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:47:43.842246 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e2e8a1-bf65-476a-a4d0-3a820ff58162 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:47:43.842246 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e2e8a1-bf65-476a-a4d0-3a820ff58162 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:47:43.862230 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-75e2e8a1-bf65-476a-a4d0-3a820ff58162 None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:47:43.863563 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-75e2e8a1-bf65-476a-a4d0-3a820ff58162 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:47:43.864375 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.auth.core [None req-12c1076d-b5cf-420b-a2e6-9b71b11a0bfc None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80312) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:47:43.865556 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-12c1076d-b5cf-420b-a2e6-9b71b11a0bfc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:47:43.923318 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1335/2684] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 1 16:47:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 16:47:43.926280 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1350/2685] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 1 16:47:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 01 16:47:43.932192 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab42996e-1db6-4656-8369-86e9d25f0374 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:47:43.932487 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab42996e-1db6-4656-8369-86e9d25f0374 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:47:43.932768 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab42996e-1db6-4656-8369-86e9d25f0374 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:47:43.933580 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686250b0-7eb9-429b-aec8-e4ad9f52f10f None None] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:47:43.933870 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686250b0-7eb9-429b-aec8-e4ad9f52f10f None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:47:43.934148 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686250b0-7eb9-429b-aec8-e4ad9f52f10f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:47:43.951888 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-ab42996e-1db6-4656-8369-86e9d25f0374 None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:47:43.953100 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-ab42996e-1db6-4656-8369-86e9d25f0374 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:47:43.953409 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.auth.core [None req-686250b0-7eb9-429b-aec8-e4ad9f52f10f None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80312) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:47:43.955547 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-686250b0-7eb9-429b-aec8-e4ad9f52f10f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:47:43.962023 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1336/2686] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 1 16:47:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 16:47:43.963277 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1351/2687] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 1 16:47:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 01 16:47:43.966246 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab42996e-1db6-4656-8369-86e9d25f0374 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:47:43.967306 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686250b0-7eb9-429b-aec8-e4ad9f52f10f None None] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:47:43.977406 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686250b0-7eb9-429b-aec8-e4ad9f52f10f None None] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:47:43.978325 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab42996e-1db6-4656-8369-86e9d25f0374 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:47:43.982209 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e313088-5002-4de8-b405-65ff202299bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:47:43.982982 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e313088-5002-4de8-b405-65ff202299bf admin admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:47:43.983266 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e313088-5002-4de8-b405-65ff202299bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:47:43.983542 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e313088-5002-4de8-b405-65ff202299bf admin admin] PATH_INFO: `/v3/domains` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:47:43.983771 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7174f3ae-a8e6-4518-9b81-90736dc783d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:47:43.984270 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e313088-5002-4de8-b405-65ff202299bf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80312) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:47:43.984437 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7174f3ae-a8e6-4518-9b81-90736dc783d7 admin admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:47:43.984609 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e313088-5002-4de8-b405-65ff202299bf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:47:43.984792 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7174f3ae-a8e6-4518-9b81-90736dc783d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:47:43.985165 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7174f3ae-a8e6-4518-9b81-90736dc783d7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:47:43.986304 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e313088-5002-4de8-b405-65ff202299bf admin admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:47:43.986743 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7174f3ae-a8e6-4518-9b81-90736dc783d7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:47:43.987266 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7174f3ae-a8e6-4518-9b81-90736dc783d7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:47:43.989000 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7174f3ae-a8e6-4518-9b81-90736dc783d7 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:47:44.000107 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1352/2688] 10.222.0.57 () {58 vars in 1152 bytes} [Fri Jul 1 16:47:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 01 16:47:44.003537 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1337/2689] 10.222.0.57 () {58 vars in 1152 bytes} [Fri Jul 1 16:47:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 01 16:47:44.006016 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e313088-5002-4de8-b405-65ff202299bf admin admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:47:44.007978 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7174f3ae-a8e6-4518-9b81-90736dc783d7 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:47:44.016442 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e313088-5002-4de8-b405-65ff202299bf admin admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:47:44.020800 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7174f3ae-a8e6-4518-9b81-90736dc783d7 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:47:44.024833 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af10cf89-6ce5-482f-89f1-e1f7a829e61d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:47:44.025708 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af10cf89-6ce5-482f-89f1-e1f7a829e61d admin admin] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:47:44.026003 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af10cf89-6ce5-482f-89f1-e1f7a829e61d admin admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:47:44.026287 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af10cf89-6ce5-482f-89f1-e1f7a829e61d admin admin] PATH_INFO: `/v3/projects` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:47:44.027040 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af10cf89-6ce5-482f-89f1-e1f7a829e61d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:47:44.027732 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ec5aed-24dd-4f75-bdcc-c91b80d7ac27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:47:44.028397 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ec5aed-24dd-4f75-bdcc-c91b80d7ac27 admin admin] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:47:44.028685 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ec5aed-24dd-4f75-bdcc-c91b80d7ac27 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:47:44.028968 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ec5aed-24dd-4f75-bdcc-c91b80d7ac27 admin admin] PATH_INFO: `/v3/projects` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:47:44.029072 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af10cf89-6ce5-482f-89f1-e1f7a829e61d admin admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:47:44.029662 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ec5aed-24dd-4f75-bdcc-c91b80d7ac27 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:47:44.031593 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ec5aed-24dd-4f75-bdcc-c91b80d7ac27 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:47:44.057873 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1353/2690] 10.222.0.57 () {60 vars in 1152 bytes} [Fri Jul 1 16:47:44 2022] POST /identity/v3/projects => generated 377 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 01 16:47:44.061929 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af10cf89-6ce5-482f-89f1-e1f7a829e61d admin admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:47:44.063990 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1338/2691] 10.222.0.57 () {60 vars in 1152 bytes} [Fri Jul 1 16:47:44 2022] POST /identity/v3/projects => generated 375 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 01 16:47:44.068033 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ec5aed-24dd-4f75-bdcc-c91b80d7ac27 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:47:44.097663 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:47:44.135900 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ec5aed-24dd-4f75-bdcc-c91b80d7ac27 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:47:44.139407 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af10cf89-6ce5-482f-89f1-e1f7a829e61d admin admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:47:44.141137 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd816502-eb7a-42c9-b95d-59bae96cc4ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:47:44.142326 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd816502-eb7a-42c9-b95d-59bae96cc4ab admin admin] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:47:44.142645 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd816502-eb7a-42c9-b95d-59bae96cc4ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:47:44.142991 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd816502-eb7a-42c9-b95d-59bae96cc4ab admin admin] PATH_INFO: `/v3/users` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:47:44.143683 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd816502-eb7a-42c9-b95d-59bae96cc4ab admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:47:44.145086 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cdd5b1c-6a8b-4719-a591-a5c704b719d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:47:44.145600 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd816502-eb7a-42c9-b95d-59bae96cc4ab admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:47:44.145800 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdd5b1c-6a8b-4719-a591-a5c704b719d5 admin admin] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:47:44.146078 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdd5b1c-6a8b-4719-a591-a5c704b719d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:47:44.146384 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdd5b1c-6a8b-4719-a591-a5c704b719d5 admin admin] PATH_INFO: `/v3/users` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:47:44.147082 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cdd5b1c-6a8b-4719-a591-a5c704b719d5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:47:44.149164 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cdd5b1c-6a8b-4719-a591-a5c704b719d5 admin admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:47:44.169311 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1339/2692] 10.222.0.57 () {60 vars in 1143 bytes} [Fri Jul 1 16:47:44 2022] POST /identity/v3/users => generated 288 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 01 16:47:44.173948 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1354/2693] 10.222.0.57 () {60 vars in 1143 bytes} [Fri Jul 1 16:47:44 2022] POST /identity/v3/users => generated 289 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 16:47:44.174504 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd816502-eb7a-42c9-b95d-59bae96cc4ab admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:47:44.178956 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cdd5b1c-6a8b-4719-a591-a5c704b719d5 admin admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:47:44.181193 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=90,keystone:DELETE=5,keystone:INSERT=5 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:47:44.185635 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd816502-eb7a-42c9-b95d-59bae96cc4ab admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:47:44.192726 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1776a0ce-3193-449b-83c1-8e14ef605372 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:47:44.193704 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1776a0ce-3193-449b-83c1-8e14ef605372 admin admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:47:44.194475 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1776a0ce-3193-449b-83c1-8e14ef605372 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:47:44.195113 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1776a0ce-3193-449b-83c1-8e14ef605372 admin admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:47:44.195930 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1776a0ce-3193-449b-83c1-8e14ef605372 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:47:44.197606 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cdd5b1c-6a8b-4719-a591-a5c704b719d5 admin admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:47:44.197772 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1776a0ce-3193-449b-83c1-8e14ef605372 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:47:44.203979 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1184ac-8634-4eeb-9809-bb6c70a1df88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:47:44.205291 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1184ac-8634-4eeb-9809-bb6c70a1df88 admin admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:47:44.205576 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1184ac-8634-4eeb-9809-bb6c70a1df88 admin admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:47:44.205873 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1184ac-8634-4eeb-9809-bb6c70a1df88 admin admin] PATH_INFO: `/v3/roles` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:47:44.206633 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1184ac-8634-4eeb-9809-bb6c70a1df88 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:47:44.208487 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1184ac-8634-4eeb-9809-bb6c70a1df88 admin admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:47:44.209690 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1340/2694] 10.222.0.57 () {58 vars in 1121 bytes} [Fri Jul 1 16:47:44 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 01 16:47:44.215081 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1776a0ce-3193-449b-83c1-8e14ef605372 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:47:44.224173 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1355/2695] 10.222.0.57 () {58 vars in 1121 bytes} [Fri Jul 1 16:47:44 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 01 16:47:44.228061 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1776a0ce-3193-449b-83c1-8e14ef605372 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:47:44.229618 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1184ac-8634-4eeb-9809-bb6c70a1df88 admin admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:47:44.240536 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eefc96dc-e82f-42a8-a905-d208edd0d4c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:47:44.241386 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefc96dc-e82f-42a8-a905-d208edd0d4c2 admin admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:47:44.241513 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefc96dc-e82f-42a8-a905-d208edd0d4c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:47:44.241789 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefc96dc-e82f-42a8-a905-d208edd0d4c2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:47:44.242120 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1184ac-8634-4eeb-9809-bb6c70a1df88 admin admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:47:44.243103 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eefc96dc-e82f-42a8-a905-d208edd0d4c2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:47:44.245885 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eefc96dc-e82f-42a8-a905-d208edd0d4c2 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:47:44.247978 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6132c8-6dac-49e6-a8e1-4eba634fdae4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:47:44.248679 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6132c8-6dac-49e6-a8e1-4eba634fdae4 admin admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:47:44.248941 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6132c8-6dac-49e6-a8e1-4eba634fdae4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:47:44.249252 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6132c8-6dac-49e6-a8e1-4eba634fdae4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:47:44.249991 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad6132c8-6dac-49e6-a8e1-4eba634fdae4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:47:44.251708 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad6132c8-6dac-49e6-a8e1-4eba634fdae4 admin admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:47:44.257625 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1341/2696] 10.222.0.57 () {58 vars in 1121 bytes} [Fri Jul 1 16:47:44 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 01 16:47:44.261877 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eefc96dc-e82f-42a8-a905-d208edd0d4c2 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:47:44.271384 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1356/2697] 10.222.0.57 () {58 vars in 1121 bytes} [Fri Jul 1 16:47:44 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 16:47:44.272789 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6132c8-6dac-49e6-a8e1-4eba634fdae4 admin admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:47:44.275458 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eefc96dc-e82f-42a8-a905-d208edd0d4c2 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:47:44.280859 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40284cb1-eca3-4d53-a047-3ce3f1844f2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:47:44.281821 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40284cb1-eca3-4d53-a047-3ce3f1844f2b admin admin] REQUEST_METHOD: `PUT` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:47:44.283062 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40284cb1-eca3-4d53-a047-3ce3f1844f2b admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:47:44.283188 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40284cb1-eca3-4d53-a047-3ce3f1844f2b admin admin] PATH_INFO: `/v3/projects/76738dc31db0434c822cf795391e6319/users/0ae8ec952ab941f987aa1e230228208c/roles/983be40bdab04c478eaa90c6b6a46b14` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:47:44.291370 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6132c8-6dac-49e6-a8e1-4eba634fdae4 admin admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:47:44.296314 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c119a143-37cd-4367-a974-c0600d415ea4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:47:44.297216 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c119a143-37cd-4367-a974-c0600d415ea4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:47:44.297501 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c119a143-37cd-4367-a974-c0600d415ea4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:47:44.297835 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c119a143-37cd-4367-a974-c0600d415ea4 admin admin] PATH_INFO: `/v3/projects/b1b152a7ad4d44f8a292995a1598d238/users/f25bcb63e6194aa9884e589db9f7ede6/roles/983be40bdab04c478eaa90c6b6a46b14` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:47:44.304150 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40284cb1-eca3-4d53-a047-3ce3f1844f2b admin admin] RBAC: Authorizing `identity:create_grant(project_id=76738dc31db0434c822cf795391e6319, user_id=0ae8ec952ab941f987aa1e230228208c, role_id=983be40bdab04c478eaa90c6b6a46b14)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:47:44.307789 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40284cb1-eca3-4d53-a047-3ce3f1844f2b admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:47:44.316881 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c119a143-37cd-4367-a974-c0600d415ea4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b1b152a7ad4d44f8a292995a1598d238, user_id=f25bcb63e6194aa9884e589db9f7ede6, role_id=983be40bdab04c478eaa90c6b6a46b14)` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:47:44.320589 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c119a143-37cd-4367-a974-c0600d415ea4 admin admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:47:44.325256 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1342/2698] 10.222.0.57 () {60 vars in 1482 bytes} [Fri Jul 1 16:47:44 2022] PUT /identity/v3/projects/76738dc31db0434c822cf795391e6319/users/0ae8ec952ab941f987aa1e230228208c/roles/983be40bdab04c478eaa90c6b6a46b14 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 01 16:47:44.340829 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34fcd42-038a-40f6-b736-61642b5795c1 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:47:44.341183 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34fcd42-038a-40f6-b736-61642b5795c1 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:47:44.341761 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34fcd42-038a-40f6-b736-61642b5795c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:47:44.343940 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1357/2699] 10.222.0.57 () {60 vars in 1482 bytes} [Fri Jul 1 16:47:44 2022] PUT /identity/v3/projects/b1b152a7ad4d44f8a292995a1598d238/users/f25bcb63e6194aa9884e589db9f7ede6/roles/983be40bdab04c478eaa90c6b6a46b14 => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 16:47:44.353351 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de4d048-ad11-4d3a-ae97-571a7fe2aafd None None] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:47:44.353651 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de4d048-ad11-4d3a-ae97-571a7fe2aafd None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:47:44.353936 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de4d048-ad11-4d3a-ae97-571a7fe2aafd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:47:44.374014 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.auth.core [None req-1de4d048-ad11-4d3a-ae97-571a7fe2aafd None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80312) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:47:44.374247 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-f34fcd42-038a-40f6-b736-61642b5795c1 None None] MFA Rules not processed for user `0ae8ec952ab941f987aa1e230228208c`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:47:44.375413 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-f34fcd42-038a-40f6-b736-61642b5795c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:47:44.375675 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-1de4d048-ad11-4d3a-ae97-571a7fe2aafd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:47:44.463678 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1358/2700] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 1 16:47:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 16:47:44.476883 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de4d048-ad11-4d3a-ae97-571a7fe2aafd None None] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:47:44.478421 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-1de4d048-ad11-4d3a-ae97-571a7fe2aafd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:47:44.480351 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1343/2701] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 1 16:47:44 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 01 16:47:44.494542 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34fcd42-038a-40f6-b736-61642b5795c1 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:47:44.496537 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-f34fcd42-038a-40f6-b736-61642b5795c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:47:44.555085 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de4d048-ad11-4d3a-ae97-571a7fe2aafd None None] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:47:44.559657 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13bb4bec-49b5-45b5-86d1-6597a35d77d0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ed200ce29684c5f80354fa9a8638fe2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0788e6725a84e57a4174284c1845755', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:47:44.560748 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bb4bec-49b5-45b5-86d1-6597a35d77d0 service neutron] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:47:44.561059 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bb4bec-49b5-45b5-86d1-6597a35d77d0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:47:44.561619 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bb4bec-49b5-45b5-86d1-6597a35d77d0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:47:44.574808 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13bb4bec-49b5-45b5-86d1-6597a35d77d0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:47:44.576930 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13bb4bec-49b5-45b5-86d1-6597a35d77d0 service neutron] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:47:44.578208 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34fcd42-038a-40f6-b736-61642b5795c1 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:47:44.585628 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915636a4-6a79-4682-a9c9-9e667c2e8b61 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbddf971a2354bbdbb9da1d63f3abf88', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0788e6725a84e57a4174284c1845755', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:47:44.587748 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915636a4-6a79-4682-a9c9-9e667c2e8b61 service cinder] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:47:44.588039 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915636a4-6a79-4682-a9c9-9e667c2e8b61 service cinder] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:47:44.588319 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915636a4-6a79-4682-a9c9-9e667c2e8b61 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:47:44.589688 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1359/2702] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 1 16:47:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 16:47:44.603684 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-915636a4-6a79-4682-a9c9-9e667c2e8b61 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:47:44.605587 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-915636a4-6a79-4682-a9c9-9e667c2e8b61 service cinder] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:47:44.624379 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1344/2703] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 1 16:47:44 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 16:47:49.215233 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0a56b0-5a23-455e-af47-ece054915bd9 None None] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:47:49.215708 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0a56b0-5a23-455e-af47-ece054915bd9 None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:47:49.215793 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0a56b0-5a23-455e-af47-ece054915bd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:47:49.269953 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.auth.core [None req-da0a56b0-5a23-455e-af47-ece054915bd9 None None] MFA Rules not processed for user `f25bcb63e6194aa9884e589db9f7ede6`. Rule list: `[]` (Enabled: `True`). {{(pid=80312) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:47:49.271255 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-da0a56b0-5a23-455e-af47-ece054915bd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:47:49.275651 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:47:49.379109 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1360/2704] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 1 16:47:49 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 16:47:49.392619 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915636a4-6a79-4682-a9c9-9e667c2e8b61 service cinder] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:47:49.394253 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-915636a4-6a79-4682-a9c9-9e667c2e8b61 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:47:49.479297 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:47:49.513527 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915636a4-6a79-4682-a9c9-9e667c2e8b61 service cinder] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:47:49.523672 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da78259d-59f4-4698-b7d8-2ab3276b11eb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a50f745b31941b7a7265c0fc852f48e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0788e6725a84e57a4174284c1845755', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:47:49.524721 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da78259d-59f4-4698-b7d8-2ab3276b11eb service nova] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:47:49.525170 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da78259d-59f4-4698-b7d8-2ab3276b11eb service nova] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:47:49.525669 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da78259d-59f4-4698-b7d8-2ab3276b11eb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:47:49.541016 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da78259d-59f4-4698-b7d8-2ab3276b11eb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:47:49.542882 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da78259d-59f4-4698-b7d8-2ab3276b11eb service nova] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:47:49.555717 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1345/2705] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 1 16:47:49 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 01 16:47:51.409405 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65c544b-ad00-4104-ae60-73c85d075bbd None None] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:47:51.409892 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65c544b-ad00-4104-ae60-73c85d075bbd None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:47:51.409992 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65c544b-ad00-4104-ae60-73c85d075bbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:47:51.431605 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.auth.core [None req-a65c544b-ad00-4104-ae60-73c85d075bbd None None] MFA Rules not processed for user `f25bcb63e6194aa9884e589db9f7ede6`. Rule list: `[]` (Enabled: `True`). {{(pid=80312) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:47:51.432833 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-a65c544b-ad00-4104-ae60-73c85d075bbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:47:51.449569 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1361/2706] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 1 16:47:51 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 16:47:55.822247 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da78259d-59f4-4698-b7d8-2ab3276b11eb service nova] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:47:55.834908 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da78259d-59f4-4698-b7d8-2ab3276b11eb service nova] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:47:55.841935 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1690df6-e3a4-4561-911f-22df28f96032 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:47:55.843007 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1690df6-e3a4-4561-911f-22df28f96032 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:47:55.843363 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1690df6-e3a4-4561-911f-22df28f96032 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:47:55.843711 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1690df6-e3a4-4561-911f-22df28f96032 admin admin] PATH_INFO: `/v3/users/0ae8ec952ab941f987aa1e230228208c` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:47:55.846459 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1690df6-e3a4-4561-911f-22df28f96032 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0ae8ec952ab941f987aa1e230228208c)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:47:55.848488 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1690df6-e3a4-4561-911f-22df28f96032 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:47:55.902951 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-e1690df6-e3a4-4561-911f-22df28f96032 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0ae8ec952ab941f987aa1e230228208c'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:47:55.903863 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-e1690df6-e3a4-4561-911f-22df28f96032 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0ae8ec952ab941f987aa1e230228208c'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:47:55.904606 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-e1690df6-e3a4-4561-911f-22df28f96032 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0ae8ec952ab941f987aa1e230228208c'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:47:55.915920 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-e1690df6-e3a4-4561-911f-22df28f96032 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0ae8ec952ab941f987aa1e230228208c'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:47:55.934670 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-e1690df6-e3a4-4561-911f-22df28f96032 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0ae8ec952ab941f987aa1e230228208c'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:47:55.947195 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1346/2707] 10.222.0.57 () {58 vars in 1223 bytes} [Fri Jul 1 16:47:55 2022] DELETE /identity/v3/users/0ae8ec952ab941f987aa1e230228208c => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 16:47:55.951853 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e7d2f0-8c02-436d-8f3e-87dca2db63f7 None None] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:47:55.952672 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e7d2f0-8c02-436d-8f3e-87dca2db63f7 None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:47:55.952955 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e7d2f0-8c02-436d-8f3e-87dca2db63f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:47:55.971388 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.auth.core [None req-12e7d2f0-8c02-436d-8f3e-87dca2db63f7 None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80312) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:47:55.972590 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-12e7d2f0-8c02-436d-8f3e-87dca2db63f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:47:56.037689 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1362/2708] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 1 16:47:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 16:47:56.047953 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1690df6-e3a4-4561-911f-22df28f96032 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:47:56.049466 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-e1690df6-e3a4-4561-911f-22df28f96032 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:47:56.108697 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1690df6-e3a4-4561-911f-22df28f96032 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:47:56.112622 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca92ba07-10f2-4b14-9243-78ec45faa86b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ed200ce29684c5f80354fa9a8638fe2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0788e6725a84e57a4174284c1845755', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:47:56.113279 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca92ba07-10f2-4b14-9243-78ec45faa86b service neutron] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:47:56.113568 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca92ba07-10f2-4b14-9243-78ec45faa86b service neutron] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:47:56.113858 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca92ba07-10f2-4b14-9243-78ec45faa86b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:47:56.123811 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca92ba07-10f2-4b14-9243-78ec45faa86b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:47:56.125269 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca92ba07-10f2-4b14-9243-78ec45faa86b service neutron] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:47:56.135808 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1347/2709] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 1 16:47:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 16:47:56.418795 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e7d2f0-8c02-436d-8f3e-87dca2db63f7 None None] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:47:56.420272 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-12e7d2f0-8c02-436d-8f3e-87dca2db63f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:47:56.433422 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e7d2f0-8c02-436d-8f3e-87dca2db63f7 None None] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:47:56.439756 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3f950f-87d3-4144-810e-d55fba6a3e5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:47:56.440799 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3f950f-87d3-4144-810e-d55fba6a3e5e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:47:56.441179 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3f950f-87d3-4144-810e-d55fba6a3e5e admin admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:47:56.441369 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3f950f-87d3-4144-810e-d55fba6a3e5e admin admin] PATH_INFO: `/v3/projects/76738dc31db0434c822cf795391e6319` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:47:56.443561 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd3f950f-87d3-4144-810e-d55fba6a3e5e admin admin] RBAC: Authorizing `identity:delete_project(project_id=76738dc31db0434c822cf795391e6319)` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:47:56.445493 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd3f950f-87d3-4144-810e-d55fba6a3e5e admin admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:47:56.495301 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-fd3f950f-87d3-4144-810e-d55fba6a3e5e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '76738dc31db0434c822cf795391e6319', 'request_id': 'req-fd3f950f-87d3-4144-810e-d55fba6a3e5e'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:47:56.504288 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-fd3f950f-87d3-4144-810e-d55fba6a3e5e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '76738dc31db0434c822cf795391e6319', 'request_id': 'req-fd3f950f-87d3-4144-810e-d55fba6a3e5e'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:47:56.513622 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-fd3f950f-87d3-4144-810e-d55fba6a3e5e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '76738dc31db0434c822cf795391e6319', 'request_id': 'req-fd3f950f-87d3-4144-810e-d55fba6a3e5e'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:47:56.527442 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-fd3f950f-87d3-4144-810e-d55fba6a3e5e admin admin] The token cache is being invalidate because project 76738dc31db0434c822cf795391e6319 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80312) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 01 16:47:56.527731 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-fd3f950f-87d3-4144-810e-d55fba6a3e5e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:47:56.528426 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-fd3f950f-87d3-4144-810e-d55fba6a3e5e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:47:56.530464 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1363/2710] 10.222.0.57 () {58 vars in 1232 bytes} [Fri Jul 1 16:47:56 2022] DELETE /identity/v3/projects/76738dc31db0434c822cf795391e6319 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 16:47:56.535757 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9b3c3f-55aa-49be-ba3e-b738eb095e4c None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:47:56.536622 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9b3c3f-55aa-49be-ba3e-b738eb095e4c None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:47:56.536622 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9b3c3f-55aa-49be-ba3e-b738eb095e4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:47:56.556322 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-7c9b3c3f-55aa-49be-ba3e-b738eb095e4c None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:47:56.557662 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-7c9b3c3f-55aa-49be-ba3e-b738eb095e4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:47:56.605128 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:47:56.622281 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1348/2711] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 1 16:47:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 01 16:47:56.630731 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab04245f-518c-430e-af62-07d0c4e9120a None None] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:47:56.631007 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab04245f-518c-430e-af62-07d0c4e9120a None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:47:56.631294 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab04245f-518c-430e-af62-07d0c4e9120a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:47:56.650012 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.auth.core [None req-ab04245f-518c-430e-af62-07d0c4e9120a None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80312) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:47:56.651256 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-ab04245f-518c-430e-af62-07d0c4e9120a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:47:56.659904 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1364/2712] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 1 16:47:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 16:47:56.664250 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9b3c3f-55aa-49be-ba3e-b738eb095e4c None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:47:56.674687 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9b3c3f-55aa-49be-ba3e-b738eb095e4c None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:47:56.679274 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e773d6c-5c46-4a66-b712-4b770bde458a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:47:56.679924 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e773d6c-5c46-4a66-b712-4b770bde458a admin admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:47:56.680209 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e773d6c-5c46-4a66-b712-4b770bde458a admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:47:56.680488 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e773d6c-5c46-4a66-b712-4b770bde458a admin admin] PATH_INFO: `/v3/domains` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:47:56.681211 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e773d6c-5c46-4a66-b712-4b770bde458a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:47:56.681519 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e773d6c-5c46-4a66-b712-4b770bde458a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:47:56.683168 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e773d6c-5c46-4a66-b712-4b770bde458a admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:47:56.694786 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1349/2713] 10.222.0.57 () {58 vars in 1152 bytes} [Fri Jul 1 16:47:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:47:56.699103 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab04245f-518c-430e-af62-07d0c4e9120a None None] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:47:56.708626 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab04245f-518c-430e-af62-07d0c4e9120a None None] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:47:56.713803 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7be3e36-fc92-4088-8ff1-320a46420bce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:47:56.714495 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7be3e36-fc92-4088-8ff1-320a46420bce admin admin] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:47:56.714785 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7be3e36-fc92-4088-8ff1-320a46420bce admin admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:47:56.715062 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7be3e36-fc92-4088-8ff1-320a46420bce admin admin] PATH_INFO: `/v3/projects` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:47:56.715767 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7be3e36-fc92-4088-8ff1-320a46420bce admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:47:56.717692 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7be3e36-fc92-4088-8ff1-320a46420bce admin admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:47:56.731474 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:47:56.745783 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1365/2714] 10.222.0.57 () {60 vars in 1152 bytes} [Fri Jul 1 16:47:56 2022] POST /identity/v3/projects => generated 375 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 01 16:47:56.750003 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e773d6c-5c46-4a66-b712-4b770bde458a admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:47:56.760199 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e773d6c-5c46-4a66-b712-4b770bde458a admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:47:56.818650 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7253c8-0095-4fa0-ab60-129d9f07066b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:47:56.819321 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7253c8-0095-4fa0-ab60-129d9f07066b admin admin] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:47:56.819604 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7253c8-0095-4fa0-ab60-129d9f07066b admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:47:56.819876 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7253c8-0095-4fa0-ab60-129d9f07066b admin admin] PATH_INFO: `/v3/users` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:47:56.820594 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d7253c8-0095-4fa0-ab60-129d9f07066b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:47:56.822548 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d7253c8-0095-4fa0-ab60-129d9f07066b admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:47:56.842597 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1350/2715] 10.222.0.57 () {60 vars in 1143 bytes} [Fri Jul 1 16:47:56 2022] POST /identity/v3/users => generated 288 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 16:47:56.847644 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7be3e36-fc92-4088-8ff1-320a46420bce admin admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:47:56.857854 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7be3e36-fc92-4088-8ff1-320a46420bce admin admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:47:56.862758 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2aa6671-8735-4daf-83e1-6f0c3558fed1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:47:56.863510 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2aa6671-8735-4daf-83e1-6f0c3558fed1 admin admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:47:56.863794 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2aa6671-8735-4daf-83e1-6f0c3558fed1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:47:56.864068 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2aa6671-8735-4daf-83e1-6f0c3558fed1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:47:56.864907 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2aa6671-8735-4daf-83e1-6f0c3558fed1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:47:56.866528 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2aa6671-8735-4daf-83e1-6f0c3558fed1 admin admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:47:56.876952 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1366/2716] 10.222.0.57 () {58 vars in 1121 bytes} [Fri Jul 1 16:47:56 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 16:47:56.880988 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7253c8-0095-4fa0-ab60-129d9f07066b admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:47:56.891699 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7253c8-0095-4fa0-ab60-129d9f07066b admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:47:56.897971 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f06834-c01a-42f5-a686-17db8647a4fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:47:56.898653 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f06834-c01a-42f5-a686-17db8647a4fd admin admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:47:56.898941 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f06834-c01a-42f5-a686-17db8647a4fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:47:56.899222 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f06834-c01a-42f5-a686-17db8647a4fd admin admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:47:56.899950 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f06834-c01a-42f5-a686-17db8647a4fd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:47:56.901564 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f06834-c01a-42f5-a686-17db8647a4fd admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:47:56.912249 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1351/2717] 10.222.0.57 () {58 vars in 1121 bytes} [Fri Jul 1 16:47:56 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 16:47:56.916414 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2aa6671-8735-4daf-83e1-6f0c3558fed1 admin admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:47:56.926372 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2aa6671-8735-4daf-83e1-6f0c3558fed1 admin admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:47:56.931182 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dfac5d5-6d4e-464e-b4e4-93e91fe5df7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:47:56.932054 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfac5d5-6d4e-464e-b4e4-93e91fe5df7b admin admin] REQUEST_METHOD: `PUT` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:47:56.932332 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfac5d5-6d4e-464e-b4e4-93e91fe5df7b admin admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:47:56.932605 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfac5d5-6d4e-464e-b4e4-93e91fe5df7b admin admin] PATH_INFO: `/v3/projects/bec339eeee9c43bb8126fe9dd12173cb/users/3fe7a1f541e44b44bfddb4b33cae144b/roles/983be40bdab04c478eaa90c6b6a46b14` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:47:56.947108 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dfac5d5-6d4e-464e-b4e4-93e91fe5df7b admin admin] RBAC: Authorizing `identity:create_grant(project_id=bec339eeee9c43bb8126fe9dd12173cb, user_id=3fe7a1f541e44b44bfddb4b33cae144b, role_id=983be40bdab04c478eaa90c6b6a46b14)` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:47:56.949724 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dfac5d5-6d4e-464e-b4e4-93e91fe5df7b admin admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:47:56.964398 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1367/2718] 10.222.0.57 () {60 vars in 1482 bytes} [Fri Jul 1 16:47:56 2022] PUT /identity/v3/projects/bec339eeee9c43bb8126fe9dd12173cb/users/3fe7a1f541e44b44bfddb4b33cae144b/roles/983be40bdab04c478eaa90c6b6a46b14 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 16:47:56.975401 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41ed476-379f-4726-bcad-c3652884c38a None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:47:56.975697 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41ed476-379f-4726-bcad-c3652884c38a None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:47:56.975981 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41ed476-379f-4726-bcad-c3652884c38a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:47:57.003137 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-d41ed476-379f-4726-bcad-c3652884c38a None None] MFA Rules not processed for user `3fe7a1f541e44b44bfddb4b33cae144b`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:47:57.005049 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-d41ed476-379f-4726-bcad-c3652884c38a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:47:57.073659 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1352/2719] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 1 16:47:56 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 16:47:57.087193 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dfac5d5-6d4e-464e-b4e4-93e91fe5df7b admin admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:47:57.089397 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-8dfac5d5-6d4e-464e-b4e4-93e91fe5df7b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:47:57.152460 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dfac5d5-6d4e-464e-b4e4-93e91fe5df7b admin admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:47:57.158477 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff04557d-2dbf-47ce-8f38-fddcc698bf36 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbddf971a2354bbdbb9da1d63f3abf88', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0788e6725a84e57a4174284c1845755', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:47:57.159577 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff04557d-2dbf-47ce-8f38-fddcc698bf36 service cinder] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:47:57.159881 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff04557d-2dbf-47ce-8f38-fddcc698bf36 service cinder] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:47:57.160126 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff04557d-2dbf-47ce-8f38-fddcc698bf36 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:47:57.169286 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff04557d-2dbf-47ce-8f38-fddcc698bf36 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:47:57.170862 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff04557d-2dbf-47ce-8f38-fddcc698bf36 service cinder] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:47:57.180298 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1368/2720] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 1 16:47:57 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 16:47:58.558689 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d539fc58-6344-43da-9381-48153b391f9a None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:47:58.559202 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d539fc58-6344-43da-9381-48153b391f9a None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:47:58.559299 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d539fc58-6344-43da-9381-48153b391f9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:47:58.577291 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-d539fc58-6344-43da-9381-48153b391f9a None None] MFA Rules not processed for user `3fe7a1f541e44b44bfddb4b33cae144b`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:47:58.578884 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-d539fc58-6344-43da-9381-48153b391f9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:47:58.586243 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1353/2721] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 1 16:47:58 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 16:48:07.176073 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:INSERT=2,keystone:SELECT=43 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:48:08.572233 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=79,keystone:INSERT=3 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:48:15.028111 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac7973a-f4fc-4cb7-8774-f68522182dfc None None] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:48:15.028619 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac7973a-f4fc-4cb7-8774-f68522182dfc None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:48:15.028731 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac7973a-f4fc-4cb7-8774-f68522182dfc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:48:15.047096 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.auth.core [None req-6ac7973a-f4fc-4cb7-8774-f68522182dfc None None] MFA Rules not processed for user `c4de1b1eeaed4fc5ae7200b2c7a90957`. Rule list: `[]` (Enabled: `True`). {{(pid=80312) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:48:15.048288 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-6ac7973a-f4fc-4cb7-8774-f68522182dfc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:48:15.103935 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1369/2722] 10.222.0.57 () {58 vars in 1009 bytes} [Fri Jul 1 16:48:15 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 16:48:15.148837 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d539fc58-6344-43da-9381-48153b391f9a None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:48:15.150233 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-d539fc58-6344-43da-9381-48153b391f9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:48:15.220674 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d539fc58-6344-43da-9381-48153b391f9a None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:48:15.226109 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f422e82-4f58-4b61-80d6-f40c6cd11fba service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '859fe410c2d2450fb803c6244839f27c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0788e6725a84e57a4174284c1845755', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:48:15.226786 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f422e82-4f58-4b61-80d6-f40c6cd11fba service swift] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:48:15.227077 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f422e82-4f58-4b61-80d6-f40c6cd11fba service swift] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:48:15.227360 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f422e82-4f58-4b61-80d6-f40c6cd11fba service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:48:15.235465 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f422e82-4f58-4b61-80d6-f40c6cd11fba service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:48:15.237300 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f422e82-4f58-4b61-80d6-f40c6cd11fba service swift] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:48:15.245383 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1354/2723] 10.222.0.57 () {60 vars in 1384 bytes} [Fri Jul 1 16:48:15 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 01 16:48:15.753821 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6048e3-a4f1-4aad-8ec3-8187f407d2b2 None None] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:48:15.754126 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6048e3-a4f1-4aad-8ec3-8187f407d2b2 None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:48:15.754457 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6048e3-a4f1-4aad-8ec3-8187f407d2b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:48:15.772126 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.auth.core [None req-ad6048e3-a4f1-4aad-8ec3-8187f407d2b2 None None] MFA Rules not processed for user `c4de1b1eeaed4fc5ae7200b2c7a90957`. Rule list: `[]` (Enabled: `True`). {{(pid=80312) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:48:15.773313 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-ad6048e3-a4f1-4aad-8ec3-8187f407d2b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:48:15.780395 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1370/2724] 10.222.0.57 () {58 vars in 967 bytes} [Fri Jul 1 16:48:15 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 01 16:48:15.792409 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f422e82-4f58-4b61-80d6-f40c6cd11fba service swift] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:48:15.803904 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f422e82-4f58-4b61-80d6-f40c6cd11fba service swift] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:48:15.809024 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126eb38a-d59d-42b0-afb8-137441139a9c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '859fe410c2d2450fb803c6244839f27c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0788e6725a84e57a4174284c1845755', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:48:15.809659 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126eb38a-d59d-42b0-afb8-137441139a9c service swift] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:48:15.809944 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126eb38a-d59d-42b0-afb8-137441139a9c service swift] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:48:15.810275 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126eb38a-d59d-42b0-afb8-137441139a9c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:48:15.819294 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-126eb38a-d59d-42b0-afb8-137441139a9c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:48:15.820880 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-126eb38a-d59d-42b0-afb8-137441139a9c service swift] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:48:15.828814 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1355/2725] 10.222.0.57 () {60 vars in 1384 bytes} [Fri Jul 1 16:48:15 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 01 16:48:16.017712 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498140f0-e79a-4435-a016-b979b473c1c8 None None] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:48:16.018273 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498140f0-e79a-4435-a016-b979b473c1c8 None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:48:16.018443 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498140f0-e79a-4435-a016-b979b473c1c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:48:16.036269 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.auth.core [None req-498140f0-e79a-4435-a016-b979b473c1c8 None None] MFA Rules not processed for user `c4de1b1eeaed4fc5ae7200b2c7a90957`. Rule list: `[]` (Enabled: `True`). {{(pid=80312) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:48:16.037468 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-498140f0-e79a-4435-a016-b979b473c1c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:48:16.044627 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1371/2726] 10.222.0.57 () {58 vars in 1009 bytes} [Fri Jul 1 16:48:16 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 16:48:16.057649 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126eb38a-d59d-42b0-afb8-137441139a9c service swift] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:48:16.068547 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126eb38a-d59d-42b0-afb8-137441139a9c service swift] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:48:16.073515 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d585444-0496-42be-b152-62d40d8fe866 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '859fe410c2d2450fb803c6244839f27c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0788e6725a84e57a4174284c1845755', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:48:16.074149 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d585444-0496-42be-b152-62d40d8fe866 service swift] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:48:16.074489 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d585444-0496-42be-b152-62d40d8fe866 service swift] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:48:16.074770 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d585444-0496-42be-b152-62d40d8fe866 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:48:16.083541 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d585444-0496-42be-b152-62d40d8fe866 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:48:16.085138 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d585444-0496-42be-b152-62d40d8fe866 service swift] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:48:16.093207 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1356/2727] 10.222.0.57 () {60 vars in 1384 bytes} [Fri Jul 1 16:48:16 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 01 16:48:16.415002 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600562f0-69b9-40c0-92f9-9f45594187ea None None] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:48:16.415294 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600562f0-69b9-40c0-92f9-9f45594187ea None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:48:16.415574 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600562f0-69b9-40c0-92f9-9f45594187ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:48:16.432860 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.auth.core [None req-600562f0-69b9-40c0-92f9-9f45594187ea None None] MFA Rules not processed for user `c4de1b1eeaed4fc5ae7200b2c7a90957`. Rule list: `[]` (Enabled: `True`). {{(pid=80312) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:48:16.434066 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-600562f0-69b9-40c0-92f9-9f45594187ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:48:16.441148 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1372/2728] 10.222.0.57 () {58 vars in 967 bytes} [Fri Jul 1 16:48:16 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 27 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 16:48:16.453007 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d585444-0496-42be-b152-62d40d8fe866 service swift] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:48:16.464858 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d585444-0496-42be-b152-62d40d8fe866 service swift] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:48:16.469808 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879724a0-190c-43a0-b4ee-f31fdf7a8d86 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '859fe410c2d2450fb803c6244839f27c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0788e6725a84e57a4174284c1845755', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:48:16.470472 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879724a0-190c-43a0-b4ee-f31fdf7a8d86 service swift] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:48:16.471827 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879724a0-190c-43a0-b4ee-f31fdf7a8d86 service swift] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:48:16.472119 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879724a0-190c-43a0-b4ee-f31fdf7a8d86 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:48:16.480031 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-879724a0-190c-43a0-b4ee-f31fdf7a8d86 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:48:16.481630 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-879724a0-190c-43a0-b4ee-f31fdf7a8d86 service swift] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:48:16.490115 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1357/2729] 10.222.0.57 () {60 vars in 1384 bytes} [Fri Jul 1 16:48:16 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 01 16:48:18.542965 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600562f0-69b9-40c0-92f9-9f45594187ea None None] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:48:18.606042 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600562f0-69b9-40c0-92f9-9f45594187ea None None] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:48:18.610624 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3db596c-2abb-432d-ab05-6e63f8ca3c07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:48:18.611489 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3db596c-2abb-432d-ab05-6e63f8ca3c07 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:48:18.611775 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3db596c-2abb-432d-ab05-6e63f8ca3c07 admin admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:48:18.612105 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3db596c-2abb-432d-ab05-6e63f8ca3c07 admin admin] PATH_INFO: `/v3/users/3fe7a1f541e44b44bfddb4b33cae144b` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:48:18.613288 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3db596c-2abb-432d-ab05-6e63f8ca3c07 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3fe7a1f541e44b44bfddb4b33cae144b)` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:48:18.615286 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3db596c-2abb-432d-ab05-6e63f8ca3c07 admin admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:48:18.663332 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-e3db596c-2abb-432d-ab05-6e63f8ca3c07 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3fe7a1f541e44b44bfddb4b33cae144b'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:48:18.677058 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-e3db596c-2abb-432d-ab05-6e63f8ca3c07 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3fe7a1f541e44b44bfddb4b33cae144b'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:48:18.685604 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-e3db596c-2abb-432d-ab05-6e63f8ca3c07 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3fe7a1f541e44b44bfddb4b33cae144b'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:48:18.686394 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-e3db596c-2abb-432d-ab05-6e63f8ca3c07 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3fe7a1f541e44b44bfddb4b33cae144b'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:48:18.687018 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-e3db596c-2abb-432d-ab05-6e63f8ca3c07 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3fe7a1f541e44b44bfddb4b33cae144b'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:48:18.700564 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1373/2730] 10.222.0.57 () {58 vars in 1223 bytes} [Fri Jul 1 16:48:18 2022] DELETE /identity/v3/users/3fe7a1f541e44b44bfddb4b33cae144b => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 16:48:18.705141 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf959f6b-49f3-49bb-a6f6-0a730b1b7cba None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:48:18.706104 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf959f6b-49f3-49bb-a6f6-0a730b1b7cba None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:48:18.706220 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf959f6b-49f3-49bb-a6f6-0a730b1b7cba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:48:18.726025 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-bf959f6b-49f3-49bb-a6f6-0a730b1b7cba None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:48:18.727301 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-bf959f6b-49f3-49bb-a6f6-0a730b1b7cba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:48:18.786482 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1358/2731] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 1 16:48:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 16:48:18.796116 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3db596c-2abb-432d-ab05-6e63f8ca3c07 admin admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:48:18.797533 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-e3db596c-2abb-432d-ab05-6e63f8ca3c07 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:48:18.804198 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:48:18.861327 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3db596c-2abb-432d-ab05-6e63f8ca3c07 admin admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:48:18.865107 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6e4b1c-f8b9-4dac-bf55-6db36369bb46 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ed200ce29684c5f80354fa9a8638fe2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0788e6725a84e57a4174284c1845755', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:48:18.865772 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6e4b1c-f8b9-4dac-bf55-6db36369bb46 service neutron] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:48:18.866035 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6e4b1c-f8b9-4dac-bf55-6db36369bb46 service neutron] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:48:18.866309 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6e4b1c-f8b9-4dac-bf55-6db36369bb46 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:48:18.875169 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be6e4b1c-f8b9-4dac-bf55-6db36369bb46 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:48:18.876594 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be6e4b1c-f8b9-4dac-bf55-6db36369bb46 service neutron] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:48:18.886499 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1374/2732] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 1 16:48:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 16:48:19.062179 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf959f6b-49f3-49bb-a6f6-0a730b1b7cba None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:48:19.064261 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-bf959f6b-49f3-49bb-a6f6-0a730b1b7cba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:48:19.077123 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf959f6b-49f3-49bb-a6f6-0a730b1b7cba None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:48:19.083536 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086059c7-9c59-46b1-ab4c-dd4e6627092c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:48:19.084407 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086059c7-9c59-46b1-ab4c-dd4e6627092c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:48:19.084689 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086059c7-9c59-46b1-ab4c-dd4e6627092c admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:48:19.084958 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086059c7-9c59-46b1-ab4c-dd4e6627092c admin admin] PATH_INFO: `/v3/projects/bec339eeee9c43bb8126fe9dd12173cb` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:48:19.086519 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-086059c7-9c59-46b1-ab4c-dd4e6627092c admin admin] RBAC: Authorizing `identity:delete_project(project_id=bec339eeee9c43bb8126fe9dd12173cb)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:48:19.088421 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-086059c7-9c59-46b1-ab4c-dd4e6627092c admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:48:19.133976 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:48:19.137221 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-086059c7-9c59-46b1-ab4c-dd4e6627092c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bec339eeee9c43bb8126fe9dd12173cb', 'request_id': 'req-086059c7-9c59-46b1-ab4c-dd4e6627092c'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:48:19.145179 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-086059c7-9c59-46b1-ab4c-dd4e6627092c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bec339eeee9c43bb8126fe9dd12173cb', 'request_id': 'req-086059c7-9c59-46b1-ab4c-dd4e6627092c'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:48:19.151383 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-086059c7-9c59-46b1-ab4c-dd4e6627092c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bec339eeee9c43bb8126fe9dd12173cb', 'request_id': 'req-086059c7-9c59-46b1-ab4c-dd4e6627092c'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:48:19.163929 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-086059c7-9c59-46b1-ab4c-dd4e6627092c admin admin] The token cache is being invalidate because project bec339eeee9c43bb8126fe9dd12173cb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80311) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 01 16:48:19.164216 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-086059c7-9c59-46b1-ab4c-dd4e6627092c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:48:19.164827 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-086059c7-9c59-46b1-ab4c-dd4e6627092c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:48:19.166444 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1359/2733] 10.222.0.57 () {58 vars in 1232 bytes} [Fri Jul 1 16:48:19 2022] DELETE /identity/v3/projects/bec339eeee9c43bb8126fe9dd12173cb => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 16:48:24.042101 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6e4b1c-f8b9-4dac-bf55-6db36369bb46 service neutron] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:48:24.051693 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6e4b1c-f8b9-4dac-bf55-6db36369bb46 service neutron] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:48:24.101983 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0251dac6-f34b-4e51-ad7b-33e320ee65b8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ed200ce29684c5f80354fa9a8638fe2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0788e6725a84e57a4174284c1845755', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:48:24.102661 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0251dac6-f34b-4e51-ad7b-33e320ee65b8 service neutron] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:48:24.102948 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0251dac6-f34b-4e51-ad7b-33e320ee65b8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:48:24.103232 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0251dac6-f34b-4e51-ad7b-33e320ee65b8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:48:24.105029 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-0251dac6-f34b-4e51-ad7b-33e320ee65b8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:48:24.169713 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0251dac6-f34b-4e51-ad7b-33e320ee65b8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:48:24.171192 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0251dac6-f34b-4e51-ad7b-33e320ee65b8 service neutron] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:48:24.181125 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1375/2734] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 1 16:48:24 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 16:48:29.162647 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:DELETE=4,keystone:SELECT=7,keystone:INSERT=1 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:48:29.184153 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086059c7-9c59-46b1-ab4c-dd4e6627092c admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:48:29.185564 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-086059c7-9c59-46b1-ab4c-dd4e6627092c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:48:29.198168 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086059c7-9c59-46b1-ab4c-dd4e6627092c admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:48:29.201740 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84eb3f48-4a81-4845-bfee-a05f88e03570 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ed200ce29684c5f80354fa9a8638fe2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0788e6725a84e57a4174284c1845755', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:48:29.202416 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84eb3f48-4a81-4845-bfee-a05f88e03570 service neutron] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:48:29.202706 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84eb3f48-4a81-4845-bfee-a05f88e03570 service neutron] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:48:29.202986 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84eb3f48-4a81-4845-bfee-a05f88e03570 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:48:29.204734 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-84eb3f48-4a81-4845-bfee-a05f88e03570 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:48:29.268902 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84eb3f48-4a81-4845-bfee-a05f88e03570 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:48:29.270384 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84eb3f48-4a81-4845-bfee-a05f88e03570 service neutron] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:48:29.279934 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1360/2735] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 1 16:48:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 16:48:34.176772 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=72 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:48:35.654733 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0251dac6-f34b-4e51-ad7b-33e320ee65b8 service neutron] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:48:35.656265 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-0251dac6-f34b-4e51-ad7b-33e320ee65b8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:48:35.679196 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0251dac6-f34b-4e51-ad7b-33e320ee65b8 service neutron] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:48:35.684941 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9362788a-2ab0-4552-b3d7-78e7d1dfa037 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:48:35.686071 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9362788a-2ab0-4552-b3d7-78e7d1dfa037 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:48:35.686465 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9362788a-2ab0-4552-b3d7-78e7d1dfa037 admin admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:48:35.686922 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9362788a-2ab0-4552-b3d7-78e7d1dfa037 admin admin] PATH_INFO: `/v3/users/5b62962ac2a14ae997fb637e80129beb` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:48:35.688374 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9362788a-2ab0-4552-b3d7-78e7d1dfa037 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5b62962ac2a14ae997fb637e80129beb)` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:48:35.691944 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9362788a-2ab0-4552-b3d7-78e7d1dfa037 admin admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:48:35.759966 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-9362788a-2ab0-4552-b3d7-78e7d1dfa037 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5b62962ac2a14ae997fb637e80129beb'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:48:35.777255 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-9362788a-2ab0-4552-b3d7-78e7d1dfa037 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5b62962ac2a14ae997fb637e80129beb'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:48:35.790305 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-9362788a-2ab0-4552-b3d7-78e7d1dfa037 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5b62962ac2a14ae997fb637e80129beb'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:48:35.791078 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-9362788a-2ab0-4552-b3d7-78e7d1dfa037 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5b62962ac2a14ae997fb637e80129beb'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:48:35.791797 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-9362788a-2ab0-4552-b3d7-78e7d1dfa037 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5b62962ac2a14ae997fb637e80129beb'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:48:35.806511 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1376/2736] 10.222.0.57 () {58 vars in 1223 bytes} [Fri Jul 1 16:48:35 2022] DELETE /identity/v3/users/5b62962ac2a14ae997fb637e80129beb => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 16:48:36.061240 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84eb3f48-4a81-4845-bfee-a05f88e03570 service neutron] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:48:36.075136 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84eb3f48-4a81-4845-bfee-a05f88e03570 service neutron] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:48:36.135363 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c56ef4e-9429-4ccf-9102-16580b1d8886 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:48:36.136219 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c56ef4e-9429-4ccf-9102-16580b1d8886 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:48:36.136499 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c56ef4e-9429-4ccf-9102-16580b1d8886 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:48:36.136774 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c56ef4e-9429-4ccf-9102-16580b1d8886 admin admin] PATH_INFO: `/v3/projects/1eb9237b6e8a4b5ca032f0664d09f28e` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:48:36.137942 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c56ef4e-9429-4ccf-9102-16580b1d8886 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1eb9237b6e8a4b5ca032f0664d09f28e)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:48:36.139886 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c56ef4e-9429-4ccf-9102-16580b1d8886 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:48:36.187287 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-6c56ef4e-9429-4ccf-9102-16580b1d8886 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1eb9237b6e8a4b5ca032f0664d09f28e', 'request_id': 'req-6c56ef4e-9429-4ccf-9102-16580b1d8886'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:48:36.193242 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-6c56ef4e-9429-4ccf-9102-16580b1d8886 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1eb9237b6e8a4b5ca032f0664d09f28e', 'request_id': 'req-6c56ef4e-9429-4ccf-9102-16580b1d8886'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:48:36.199575 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-6c56ef4e-9429-4ccf-9102-16580b1d8886 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1eb9237b6e8a4b5ca032f0664d09f28e', 'request_id': 'req-6c56ef4e-9429-4ccf-9102-16580b1d8886'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:48:36.213849 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-6c56ef4e-9429-4ccf-9102-16580b1d8886 admin admin] The token cache is being invalidate because project 1eb9237b6e8a4b5ca032f0664d09f28e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80311) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 01 16:48:36.214063 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-6c56ef4e-9429-4ccf-9102-16580b1d8886 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:48:36.214688 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-6c56ef4e-9429-4ccf-9102-16580b1d8886 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:48:36.216236 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1361/2737] 10.222.0.57 () {58 vars in 1232 bytes} [Fri Jul 1 16:48:36 2022] DELETE /identity/v3/projects/1eb9237b6e8a4b5ca032f0664d09f28e => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 16:48:36.220329 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9362788a-2ab0-4552-b3d7-78e7d1dfa037 admin admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:48:36.221772 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-9362788a-2ab0-4552-b3d7-78e7d1dfa037 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:48:36.235803 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9362788a-2ab0-4552-b3d7-78e7d1dfa037 admin admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:48:36.297088 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1406ed-0606-4d2b-9721-24507e4e563d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:48:36.297959 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1406ed-0606-4d2b-9721-24507e4e563d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:48:36.298244 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1406ed-0606-4d2b-9721-24507e4e563d admin admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:48:36.298570 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1406ed-0606-4d2b-9721-24507e4e563d admin admin] PATH_INFO: `/v3/users/1cc31ac5f57b46d1ad501cfa05a6cbbc` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:48:36.299747 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a1406ed-0606-4d2b-9721-24507e4e563d admin admin] RBAC: Authorizing `identity:delete_user(user_id=1cc31ac5f57b46d1ad501cfa05a6cbbc)` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:48:36.302421 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a1406ed-0606-4d2b-9721-24507e4e563d admin admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:48:36.350232 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-5a1406ed-0606-4d2b-9721-24507e4e563d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1cc31ac5f57b46d1ad501cfa05a6cbbc'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:48:36.362464 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-5a1406ed-0606-4d2b-9721-24507e4e563d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1cc31ac5f57b46d1ad501cfa05a6cbbc'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:48:36.369934 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-5a1406ed-0606-4d2b-9721-24507e4e563d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1cc31ac5f57b46d1ad501cfa05a6cbbc'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:48:36.370865 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-5a1406ed-0606-4d2b-9721-24507e4e563d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1cc31ac5f57b46d1ad501cfa05a6cbbc'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:48:36.371502 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-5a1406ed-0606-4d2b-9721-24507e4e563d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1cc31ac5f57b46d1ad501cfa05a6cbbc'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:48:36.373219 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=80,keystone:DELETE=19,keystone:INSERT=1 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:48:36.387519 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1377/2738] 10.222.0.57 () {58 vars in 1223 bytes} [Fri Jul 1 16:48:36 2022] DELETE /identity/v3/users/1cc31ac5f57b46d1ad501cfa05a6cbbc => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 16:48:36.580080 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c56ef4e-9429-4ccf-9102-16580b1d8886 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:48:36.581588 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-6c56ef4e-9429-4ccf-9102-16580b1d8886 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:48:36.604357 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c56ef4e-9429-4ccf-9102-16580b1d8886 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:48:36.626308 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:48:36.668777 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55a1498-c192-4df2-9894-f18d333ff1f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:48:36.670117 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55a1498-c192-4df2-9894-f18d333ff1f5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:48:36.670445 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55a1498-c192-4df2-9894-f18d333ff1f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:48:36.670731 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55a1498-c192-4df2-9894-f18d333ff1f5 admin admin] PATH_INFO: `/v3/projects/03752bab4db8497a911823c2ef95017f` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:48:36.672561 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e55a1498-c192-4df2-9894-f18d333ff1f5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=03752bab4db8497a911823c2ef95017f)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:48:36.675373 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e55a1498-c192-4df2-9894-f18d333ff1f5 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:48:36.731002 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-e55a1498-c192-4df2-9894-f18d333ff1f5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '03752bab4db8497a911823c2ef95017f', 'request_id': 'req-e55a1498-c192-4df2-9894-f18d333ff1f5'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:48:36.736793 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-e55a1498-c192-4df2-9894-f18d333ff1f5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '03752bab4db8497a911823c2ef95017f', 'request_id': 'req-e55a1498-c192-4df2-9894-f18d333ff1f5'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:48:36.744097 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-e55a1498-c192-4df2-9894-f18d333ff1f5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '03752bab4db8497a911823c2ef95017f', 'request_id': 'req-e55a1498-c192-4df2-9894-f18d333ff1f5'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:48:36.757183 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-e55a1498-c192-4df2-9894-f18d333ff1f5 admin admin] The token cache is being invalidate because project 03752bab4db8497a911823c2ef95017f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80311) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 01 16:48:36.757489 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-e55a1498-c192-4df2-9894-f18d333ff1f5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:48:36.758116 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-e55a1498-c192-4df2-9894-f18d333ff1f5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:48:36.759830 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1362/2739] 10.222.0.57 () {58 vars in 1232 bytes} [Fri Jul 1 16:48:36 2022] DELETE /identity/v3/projects/03752bab4db8497a911823c2ef95017f => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 16:48:46.385631 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:INSERT=1,keystone:DELETE=1,keystone:SELECT=2 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:48:46.753797 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=40,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:49:24.666044 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1406ed-0606-4d2b-9721-24507e4e563d admin admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:49:24.667497 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-5a1406ed-0606-4d2b-9721-24507e4e563d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:49:24.737776 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1406ed-0606-4d2b-9721-24507e4e563d admin admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:49:24.797551 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3815e64e-0f90-4f49-a8b2-af65b39ef44b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc38af18dec4e9b8143df23af6aad43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0788e6725a84e57a4174284c1845755', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:49:24.798199 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3815e64e-0f90-4f49-a8b2-af65b39ef44b service placement] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:49:24.798533 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3815e64e-0f90-4f49-a8b2-af65b39ef44b service placement] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:49:24.798799 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3815e64e-0f90-4f49-a8b2-af65b39ef44b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:49:24.800566 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-3815e64e-0f90-4f49-a8b2-af65b39ef44b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:49:24.811609 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3815e64e-0f90-4f49-a8b2-af65b39ef44b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:49:24.813186 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3815e64e-0f90-4f49-a8b2-af65b39ef44b service placement] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:49:24.823746 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1378/2740] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 1 16:49:24 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 16:49:34.818789 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=58 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:49:40.172735 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbeec16-94df-40e1-aa95-b916743320b1 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:49:40.173253 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbeec16-94df-40e1-aa95-b916743320b1 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:49:40.173362 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbeec16-94df-40e1-aa95-b916743320b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:49:40.191079 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-3dbeec16-94df-40e1-aa95-b916743320b1 None None] MFA Rules not processed for user `d6589e2636dd46838a3a0c66d18e0d00`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:49:40.192261 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-3dbeec16-94df-40e1-aa95-b916743320b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:49:40.247993 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1363/2741] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 1 16:49:40 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 16:49:44.350369 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3815e64e-0f90-4f49-a8b2-af65b39ef44b service placement] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:49:44.352392 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-3815e64e-0f90-4f49-a8b2-af65b39ef44b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:49:44.429189 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3815e64e-0f90-4f49-a8b2-af65b39ef44b service placement] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:49:44.435746 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c5af85-0e49-46ab-9453-45a92adaf5e9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a50f745b31941b7a7265c0fc852f48e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0788e6725a84e57a4174284c1845755', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:49:44.436401 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c5af85-0e49-46ab-9453-45a92adaf5e9 service nova] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:49:44.436706 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c5af85-0e49-46ab-9453-45a92adaf5e9 service nova] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:49:44.437010 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c5af85-0e49-46ab-9453-45a92adaf5e9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:49:44.438838 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-e9c5af85-0e49-46ab-9453-45a92adaf5e9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:49:44.449932 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c5af85-0e49-46ab-9453-45a92adaf5e9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:49:44.451601 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c5af85-0e49-46ab-9453-45a92adaf5e9 service nova] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:49:44.461863 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1379/2742] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 1 16:49:44 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 16:49:50.244524 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=25 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:49:54.456956 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=34 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:50:36.028376 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dbeec16-94df-40e1-aa95-b916743320b1 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:50:36.029879 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-3dbeec16-94df-40e1-aa95-b916743320b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:50:36.088445 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dbeec16-94df-40e1-aa95-b916743320b1 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:50:36.092580 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118187aa-831a-4064-9420-8b4525d6c633 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ed200ce29684c5f80354fa9a8638fe2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0788e6725a84e57a4174284c1845755', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:50:36.093465 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118187aa-831a-4064-9420-8b4525d6c633 service neutron] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:50:36.093751 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118187aa-831a-4064-9420-8b4525d6c633 service neutron] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:50:36.094028 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118187aa-831a-4064-9420-8b4525d6c633 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:50:36.095929 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-118187aa-831a-4064-9420-8b4525d6c633 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:50:36.110946 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118187aa-831a-4064-9420-8b4525d6c633 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:50:36.112402 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118187aa-831a-4064-9420-8b4525d6c633 service neutron] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:50:36.123625 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1364/2743] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 1 16:50:36 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 16:50:37.297407 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c5af85-0e49-46ab-9453-45a92adaf5e9 service nova] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:50:37.307078 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c5af85-0e49-46ab-9453-45a92adaf5e9 service nova] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:50:37.310894 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662b0c4f-be1a-40f1-98df-b686e0cf0116 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ed200ce29684c5f80354fa9a8638fe2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0788e6725a84e57a4174284c1845755', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:50:37.311531 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662b0c4f-be1a-40f1-98df-b686e0cf0116 service neutron] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:50:37.311819 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662b0c4f-be1a-40f1-98df-b686e0cf0116 service neutron] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:50:37.312106 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662b0c4f-be1a-40f1-98df-b686e0cf0116 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:50:37.313866 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-662b0c4f-be1a-40f1-98df-b686e0cf0116 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:50:37.327346 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-662b0c4f-be1a-40f1-98df-b686e0cf0116 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:50:37.328773 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-662b0c4f-be1a-40f1-98df-b686e0cf0116 service neutron] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:50:37.340803 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1380/2744] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 1 16:50:37 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 16:50:43.085939 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabd6c1c-4030-4f99-8217-5006b7d1fe46 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:50:43.086671 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabd6c1c-4030-4f99-8217-5006b7d1fe46 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:50:43.086812 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabd6c1c-4030-4f99-8217-5006b7d1fe46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:50:43.105254 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-fabd6c1c-4030-4f99-8217-5006b7d1fe46 None None] MFA Rules not processed for user `f25bcb63e6194aa9884e589db9f7ede6`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:50:43.106418 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-fabd6c1c-4030-4f99-8217-5006b7d1fe46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:50:43.172797 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1365/2745] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 1 16:50:43 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 16:50:43.818457 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662b0c4f-be1a-40f1-98df-b686e0cf0116 service neutron] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:50:43.819819 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-662b0c4f-be1a-40f1-98df-b686e0cf0116 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:50:43.839111 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662b0c4f-be1a-40f1-98df-b686e0cf0116 service neutron] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:50:43.849453 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945da848-a6f8-4326-a7a9-994026d9e04b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc38af18dec4e9b8143df23af6aad43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0788e6725a84e57a4174284c1845755', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:50:43.849695 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945da848-a6f8-4326-a7a9-994026d9e04b service placement] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:50:43.849987 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945da848-a6f8-4326-a7a9-994026d9e04b service placement] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:50:43.850265 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945da848-a6f8-4326-a7a9-994026d9e04b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:50:43.853332 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-945da848-a6f8-4326-a7a9-994026d9e04b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:50:43.865589 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945da848-a6f8-4326-a7a9-994026d9e04b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:50:43.867223 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945da848-a6f8-4326-a7a9-994026d9e04b service placement] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:50:43.878870 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1381/2746] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 1 16:50:43 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 16:50:46.516855 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fabd6c1c-4030-4f99-8217-5006b7d1fe46 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:50:46.526982 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fabd6c1c-4030-4f99-8217-5006b7d1fe46 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:50:46.531140 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5869ea22-8089-4e38-823a-78a011757315 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ed200ce29684c5f80354fa9a8638fe2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0788e6725a84e57a4174284c1845755', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:50:46.531798 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5869ea22-8089-4e38-823a-78a011757315 service neutron] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:50:46.532083 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5869ea22-8089-4e38-823a-78a011757315 service neutron] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:50:46.532362 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5869ea22-8089-4e38-823a-78a011757315 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:50:46.534321 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-5869ea22-8089-4e38-823a-78a011757315 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:50:46.545246 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5869ea22-8089-4e38-823a-78a011757315 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:50:46.546803 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5869ea22-8089-4e38-823a-78a011757315 service neutron] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:50:46.556216 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1366/2747] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 1 16:50:46 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 16:50:53.874207 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=12 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:50:55.744163 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945da848-a6f8-4326-a7a9-994026d9e04b service placement] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:50:55.756743 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945da848-a6f8-4326-a7a9-994026d9e04b service placement] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:50:55.763885 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a247f2-dda4-4425-9bcc-339b3376bcf9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a50f745b31941b7a7265c0fc852f48e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0788e6725a84e57a4174284c1845755', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:50:55.764558 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a247f2-dda4-4425-9bcc-339b3376bcf9 service nova] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:50:55.764831 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a247f2-dda4-4425-9bcc-339b3376bcf9 service nova] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:50:55.765113 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a247f2-dda4-4425-9bcc-339b3376bcf9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:50:55.766900 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-b8a247f2-dda4-4425-9bcc-339b3376bcf9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:50:55.779998 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a247f2-dda4-4425-9bcc-339b3376bcf9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:50:55.781601 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a247f2-dda4-4425-9bcc-339b3376bcf9 service nova] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:50:55.792998 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1382/2748] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 1 16:50:55 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 16:50:56.551586 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=57 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:51:05.787693 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=6 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:51:37.281022 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5869ea22-8089-4e38-823a-78a011757315 service neutron] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:51:37.282695 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-5869ea22-8089-4e38-823a-78a011757315 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:51:37.300283 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5869ea22-8089-4e38-823a-78a011757315 service neutron] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:51:37.307182 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77dcf2cc-a40c-4844-8382-6532593f6948 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a50f745b31941b7a7265c0fc852f48e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0788e6725a84e57a4174284c1845755', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:51:37.307875 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77dcf2cc-a40c-4844-8382-6532593f6948 service nova] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:51:37.308265 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77dcf2cc-a40c-4844-8382-6532593f6948 service nova] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:51:37.308577 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77dcf2cc-a40c-4844-8382-6532593f6948 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:51:37.310540 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-77dcf2cc-a40c-4844-8382-6532593f6948 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:51:37.323632 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77dcf2cc-a40c-4844-8382-6532593f6948 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:51:37.325393 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77dcf2cc-a40c-4844-8382-6532593f6948 service nova] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:51:37.337829 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1367/2749] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 1 16:51:37 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 16:51:47.334223 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=6 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:51:48.787131 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a247f2-dda4-4425-9bcc-339b3376bcf9 service nova] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:51:48.797276 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a247f2-dda4-4425-9bcc-339b3376bcf9 service nova] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:51:48.800878 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398e2f80-c61e-4a93-8483-e87fbb1d372a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ed200ce29684c5f80354fa9a8638fe2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0788e6725a84e57a4174284c1845755', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:51:48.801517 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398e2f80-c61e-4a93-8483-e87fbb1d372a service neutron] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:51:48.801802 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398e2f80-c61e-4a93-8483-e87fbb1d372a service neutron] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:51:48.802079 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398e2f80-c61e-4a93-8483-e87fbb1d372a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:51:48.804272 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-398e2f80-c61e-4a93-8483-e87fbb1d372a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:51:48.879150 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398e2f80-c61e-4a93-8483-e87fbb1d372a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:51:48.880607 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398e2f80-c61e-4a93-8483-e87fbb1d372a service neutron] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:51:48.893201 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1383/2750] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 1 16:51:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 16:51:52.412635 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77dcf2cc-a40c-4844-8382-6532593f6948 service nova] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:51:52.414767 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-77dcf2cc-a40c-4844-8382-6532593f6948 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:51:52.428934 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77dcf2cc-a40c-4844-8382-6532593f6948 service nova] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:51:52.434069 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4981c2-cbb9-4142-b843-9c23b17401bd service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc38af18dec4e9b8143df23af6aad43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0788e6725a84e57a4174284c1845755', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:51:52.434810 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4981c2-cbb9-4142-b843-9c23b17401bd service placement] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:51:52.435060 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4981c2-cbb9-4142-b843-9c23b17401bd service placement] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:51:52.435341 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4981c2-cbb9-4142-b843-9c23b17401bd service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:51:52.437225 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-ca4981c2-cbb9-4142-b843-9c23b17401bd service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:51:52.450179 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca4981c2-cbb9-4142-b843-9c23b17401bd service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:51:52.451807 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca4981c2-cbb9-4142-b843-9c23b17401bd service placement] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:51:52.464622 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1368/2751] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 1 16:51:52 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 16:51:55.974780 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398e2f80-c61e-4a93-8483-e87fbb1d372a service neutron] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:51:55.976972 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-398e2f80-c61e-4a93-8483-e87fbb1d372a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:51:55.999811 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398e2f80-c61e-4a93-8483-e87fbb1d372a service neutron] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:51:56.005574 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa65b9d-56cb-48c3-8f5f-05d2b15807ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:51:56.006564 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa65b9d-56cb-48c3-8f5f-05d2b15807ad admin admin] REQUEST_METHOD: `DELETE` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:51:56.007519 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa65b9d-56cb-48c3-8f5f-05d2b15807ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:51:56.007879 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa65b9d-56cb-48c3-8f5f-05d2b15807ad admin admin] PATH_INFO: `/v3/users/d6589e2636dd46838a3a0c66d18e0d00` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:51:56.009298 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa65b9d-56cb-48c3-8f5f-05d2b15807ad admin admin] RBAC: Authorizing `identity:delete_user(user_id=d6589e2636dd46838a3a0c66d18e0d00)` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:51:56.011634 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa65b9d-56cb-48c3-8f5f-05d2b15807ad admin admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:51:56.065490 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-0fa65b9d-56cb-48c3-8f5f-05d2b15807ad admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd6589e2636dd46838a3a0c66d18e0d00'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:51:56.078467 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-0fa65b9d-56cb-48c3-8f5f-05d2b15807ad admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd6589e2636dd46838a3a0c66d18e0d00'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:51:56.093169 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-0fa65b9d-56cb-48c3-8f5f-05d2b15807ad admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd6589e2636dd46838a3a0c66d18e0d00'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:51:56.093971 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-0fa65b9d-56cb-48c3-8f5f-05d2b15807ad admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd6589e2636dd46838a3a0c66d18e0d00'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:51:56.094822 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-0fa65b9d-56cb-48c3-8f5f-05d2b15807ad admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd6589e2636dd46838a3a0c66d18e0d00'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:51:56.113438 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1384/2752] 10.222.0.57 () {58 vars in 1223 bytes} [Fri Jul 1 16:51:55 2022] DELETE /identity/v3/users/d6589e2636dd46838a3a0c66d18e0d00 => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 16:51:56.316342 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4981c2-cbb9-4142-b843-9c23b17401bd service placement] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:51:56.328671 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4981c2-cbb9-4142-b843-9c23b17401bd service placement] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:51:56.405839 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20a20f3-dc23-41a5-9d1e-6876c2644f1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:51:56.406836 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20a20f3-dc23-41a5-9d1e-6876c2644f1c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:51:56.407171 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20a20f3-dc23-41a5-9d1e-6876c2644f1c admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:51:56.407468 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20a20f3-dc23-41a5-9d1e-6876c2644f1c admin admin] PATH_INFO: `/v3/projects/0076c4e65bce4bd58478d09d62607981` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:51:56.408729 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e20a20f3-dc23-41a5-9d1e-6876c2644f1c admin admin] RBAC: Authorizing `identity:delete_project(project_id=0076c4e65bce4bd58478d09d62607981)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:51:56.410929 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e20a20f3-dc23-41a5-9d1e-6876c2644f1c admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:51:56.466037 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-e20a20f3-dc23-41a5-9d1e-6876c2644f1c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0076c4e65bce4bd58478d09d62607981', 'request_id': 'req-e20a20f3-dc23-41a5-9d1e-6876c2644f1c'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:51:56.473484 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-e20a20f3-dc23-41a5-9d1e-6876c2644f1c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0076c4e65bce4bd58478d09d62607981', 'request_id': 'req-e20a20f3-dc23-41a5-9d1e-6876c2644f1c'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:51:56.480287 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-e20a20f3-dc23-41a5-9d1e-6876c2644f1c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0076c4e65bce4bd58478d09d62607981', 'request_id': 'req-e20a20f3-dc23-41a5-9d1e-6876c2644f1c'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:51:56.495306 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-e20a20f3-dc23-41a5-9d1e-6876c2644f1c admin admin] The token cache is being invalidate because project 0076c4e65bce4bd58478d09d62607981 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80311) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 01 16:51:56.495604 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-e20a20f3-dc23-41a5-9d1e-6876c2644f1c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:51:56.496336 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-e20a20f3-dc23-41a5-9d1e-6876c2644f1c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:51:56.498716 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1369/2753] 10.222.0.57 () {58 vars in 1232 bytes} [Fri Jul 1 16:51:56 2022] DELETE /identity/v3/projects/0076c4e65bce4bd58478d09d62607981 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 16:51:56.502961 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa65b9d-56cb-48c3-8f5f-05d2b15807ad admin admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:51:56.504493 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-0fa65b9d-56cb-48c3-8f5f-05d2b15807ad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:51:56.517765 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa65b9d-56cb-48c3-8f5f-05d2b15807ad admin admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:51:56.579174 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b72dbb-e507-4755-b115-609bd159856b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:51:56.580010 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b72dbb-e507-4755-b115-609bd159856b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:51:56.580293 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b72dbb-e507-4755-b115-609bd159856b admin admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:51:56.580573 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b72dbb-e507-4755-b115-609bd159856b admin admin] PATH_INFO: `/v3/users/296adde7ece444e391cedf9710323c44` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:51:56.582025 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b72dbb-e507-4755-b115-609bd159856b admin admin] RBAC: Authorizing `identity:delete_user(user_id=296adde7ece444e391cedf9710323c44)` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:51:56.584033 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b72dbb-e507-4755-b115-609bd159856b admin admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:51:56.590589 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:51:56.637253 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-d0b72dbb-e507-4755-b115-609bd159856b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '296adde7ece444e391cedf9710323c44'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:51:56.652347 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-d0b72dbb-e507-4755-b115-609bd159856b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '296adde7ece444e391cedf9710323c44'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:51:56.660770 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-d0b72dbb-e507-4755-b115-609bd159856b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '296adde7ece444e391cedf9710323c44'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:51:56.661537 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-d0b72dbb-e507-4755-b115-609bd159856b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '296adde7ece444e391cedf9710323c44'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:51:56.662152 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-d0b72dbb-e507-4755-b115-609bd159856b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '296adde7ece444e391cedf9710323c44'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:51:56.679356 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1385/2754] 10.222.0.57 () {58 vars in 1223 bytes} [Fri Jul 1 16:51:56 2022] DELETE /identity/v3/users/296adde7ece444e391cedf9710323c44 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 16:51:56.963061 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20a20f3-dc23-41a5-9d1e-6876c2644f1c admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:51:56.964569 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-e20a20f3-dc23-41a5-9d1e-6876c2644f1c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:51:56.979966 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20a20f3-dc23-41a5-9d1e-6876c2644f1c admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:51:57.046285 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e205cc53-6c1b-439f-aaa0-605d3f75e637 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:51:57.047778 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e205cc53-6c1b-439f-aaa0-605d3f75e637 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:51:57.048061 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e205cc53-6c1b-439f-aaa0-605d3f75e637 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:51:57.048378 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e205cc53-6c1b-439f-aaa0-605d3f75e637 admin admin] PATH_INFO: `/v3/projects/bb7f4747ad734029884a38be8a3adfbf` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:51:57.049661 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e205cc53-6c1b-439f-aaa0-605d3f75e637 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bb7f4747ad734029884a38be8a3adfbf)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:51:57.051826 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e205cc53-6c1b-439f-aaa0-605d3f75e637 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:51:57.075645 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:51:57.112592 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-e205cc53-6c1b-439f-aaa0-605d3f75e637 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bb7f4747ad734029884a38be8a3adfbf', 'request_id': 'req-e205cc53-6c1b-439f-aaa0-605d3f75e637'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:51:57.119146 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-e205cc53-6c1b-439f-aaa0-605d3f75e637 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bb7f4747ad734029884a38be8a3adfbf', 'request_id': 'req-e205cc53-6c1b-439f-aaa0-605d3f75e637'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:51:57.125745 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-e205cc53-6c1b-439f-aaa0-605d3f75e637 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bb7f4747ad734029884a38be8a3adfbf', 'request_id': 'req-e205cc53-6c1b-439f-aaa0-605d3f75e637'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:51:57.140609 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-e205cc53-6c1b-439f-aaa0-605d3f75e637 admin admin] The token cache is being invalidate because project bb7f4747ad734029884a38be8a3adfbf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80311) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 01 16:51:57.140893 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-e205cc53-6c1b-439f-aaa0-605d3f75e637 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:51:57.141509 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-e205cc53-6c1b-439f-aaa0-605d3f75e637 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:51:57.143286 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1370/2755] 10.222.0.57 () {58 vars in 1232 bytes} [Fri Jul 1 16:51:56 2022] DELETE /identity/v3/projects/bb7f4747ad734029884a38be8a3adfbf => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 16:52:04.016761 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b72dbb-e507-4755-b115-609bd159856b admin admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:52:04.018954 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-d0b72dbb-e507-4755-b115-609bd159856b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:52:04.035438 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b72dbb-e507-4755-b115-609bd159856b admin admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:52:04.113752 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5937c68-c218-4663-82c5-f7fad2c26abf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:52:04.114708 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5937c68-c218-4663-82c5-f7fad2c26abf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:52:04.114991 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5937c68-c218-4663-82c5-f7fad2c26abf admin admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:52:04.115267 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5937c68-c218-4663-82c5-f7fad2c26abf admin admin] PATH_INFO: `/v3/users/f25bcb63e6194aa9884e589db9f7ede6` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:52:04.117110 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5937c68-c218-4663-82c5-f7fad2c26abf admin admin] RBAC: Authorizing `identity:delete_user(user_id=f25bcb63e6194aa9884e589db9f7ede6)` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:52:04.119094 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5937c68-c218-4663-82c5-f7fad2c26abf admin admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:52:04.170741 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-c5937c68-c218-4663-82c5-f7fad2c26abf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f25bcb63e6194aa9884e589db9f7ede6'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:52:04.183899 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-c5937c68-c218-4663-82c5-f7fad2c26abf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f25bcb63e6194aa9884e589db9f7ede6'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:52:04.190949 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-c5937c68-c218-4663-82c5-f7fad2c26abf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f25bcb63e6194aa9884e589db9f7ede6'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:52:04.191625 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-c5937c68-c218-4663-82c5-f7fad2c26abf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f25bcb63e6194aa9884e589db9f7ede6'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:52:04.192254 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-c5937c68-c218-4663-82c5-f7fad2c26abf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f25bcb63e6194aa9884e589db9f7ede6'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:52:04.207471 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1386/2756] 10.222.0.57 () {58 vars in 1223 bytes} [Fri Jul 1 16:52:04 2022] DELETE /identity/v3/users/f25bcb63e6194aa9884e589db9f7ede6 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 16:52:04.393099 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e205cc53-6c1b-439f-aaa0-605d3f75e637 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:52:04.394659 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-e205cc53-6c1b-439f-aaa0-605d3f75e637 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:52:04.408344 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e205cc53-6c1b-439f-aaa0-605d3f75e637 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:52:04.467762 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d209dbf5-1419-4926-a7b2-da9318a8177b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:52:04.468617 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d209dbf5-1419-4926-a7b2-da9318a8177b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:52:04.468895 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d209dbf5-1419-4926-a7b2-da9318a8177b admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:52:04.469165 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d209dbf5-1419-4926-a7b2-da9318a8177b admin admin] PATH_INFO: `/v3/projects/b1b152a7ad4d44f8a292995a1598d238` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:52:04.470337 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d209dbf5-1419-4926-a7b2-da9318a8177b admin admin] RBAC: Authorizing `identity:delete_project(project_id=b1b152a7ad4d44f8a292995a1598d238)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:52:04.472252 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d209dbf5-1419-4926-a7b2-da9318a8177b admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:52:04.525144 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-d209dbf5-1419-4926-a7b2-da9318a8177b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b1b152a7ad4d44f8a292995a1598d238', 'request_id': 'req-d209dbf5-1419-4926-a7b2-da9318a8177b'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:52:04.530579 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-d209dbf5-1419-4926-a7b2-da9318a8177b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b1b152a7ad4d44f8a292995a1598d238', 'request_id': 'req-d209dbf5-1419-4926-a7b2-da9318a8177b'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:52:04.536780 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-d209dbf5-1419-4926-a7b2-da9318a8177b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b1b152a7ad4d44f8a292995a1598d238', 'request_id': 'req-d209dbf5-1419-4926-a7b2-da9318a8177b'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:52:04.549781 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-d209dbf5-1419-4926-a7b2-da9318a8177b admin admin] The token cache is being invalidate because project b1b152a7ad4d44f8a292995a1598d238 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80311) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 01 16:52:04.550053 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-d209dbf5-1419-4926-a7b2-da9318a8177b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:52:04.550702 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-d209dbf5-1419-4926-a7b2-da9318a8177b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:52:04.552245 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1371/2757] 10.222.0.57 () {58 vars in 1232 bytes} [Fri Jul 1 16:52:04 2022] DELETE /identity/v3/projects/b1b152a7ad4d44f8a292995a1598d238 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 16:52:04.558654 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb431ffa-6053-4553-9cea-311b02a51b6e None None] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:52:04.558945 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb431ffa-6053-4553-9cea-311b02a51b6e None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:52:04.559226 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb431ffa-6053-4553-9cea-311b02a51b6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:52:04.566742 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:52:04.578758 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.auth.core [None req-fb431ffa-6053-4553-9cea-311b02a51b6e None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80312) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:52:04.580014 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-fb431ffa-6053-4553-9cea-311b02a51b6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:52:04.639687 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1387/2758] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 1 16:52:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 16:52:04.648330 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b5ffc8-a9b5-4038-902c-5d59a2f2ff3e None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:52:04.648620 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b5ffc8-a9b5-4038-902c-5d59a2f2ff3e None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:52:04.648904 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b5ffc8-a9b5-4038-902c-5d59a2f2ff3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:52:04.668328 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-c9b5ffc8-a9b5-4038-902c-5d59a2f2ff3e None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:52:04.669535 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-c9b5ffc8-a9b5-4038-902c-5d59a2f2ff3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:52:04.677456 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1372/2759] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 1 16:52:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 16:52:04.683863 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb431ffa-6053-4553-9cea-311b02a51b6e None None] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:52:04.693737 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb431ffa-6053-4553-9cea-311b02a51b6e None None] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:52:04.699344 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1015dded-b387-4eb4-bd0d-1c833c86c12a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:52:04.700013 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1015dded-b387-4eb4-bd0d-1c833c86c12a admin admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:52:04.700301 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1015dded-b387-4eb4-bd0d-1c833c86c12a admin admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:52:04.700581 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1015dded-b387-4eb4-bd0d-1c833c86c12a admin admin] PATH_INFO: `/v3/domains` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:52:04.701368 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1015dded-b387-4eb4-bd0d-1c833c86c12a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80312) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:52:04.701674 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1015dded-b387-4eb4-bd0d-1c833c86c12a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:52:04.703312 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1015dded-b387-4eb4-bd0d-1c833c86c12a admin admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:52:04.716062 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1388/2760] 10.222.0.57 () {58 vars in 1152 bytes} [Fri Jul 1 16:52:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:52:04.720467 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b5ffc8-a9b5-4038-902c-5d59a2f2ff3e None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:52:04.729975 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b5ffc8-a9b5-4038-902c-5d59a2f2ff3e None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:52:04.734708 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9df1341-2797-4ab3-9963-81e6d680a3ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:52:04.735360 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9df1341-2797-4ab3-9963-81e6d680a3ce admin admin] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:52:04.735645 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9df1341-2797-4ab3-9963-81e6d680a3ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:52:04.735922 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9df1341-2797-4ab3-9963-81e6d680a3ce admin admin] PATH_INFO: `/v3/projects` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:52:04.736607 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9df1341-2797-4ab3-9963-81e6d680a3ce admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:52:04.738538 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9df1341-2797-4ab3-9963-81e6d680a3ce admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:52:04.768048 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1373/2761] 10.222.0.57 () {60 vars in 1152 bytes} [Fri Jul 1 16:52:04 2022] POST /identity/v3/projects => generated 405 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 16:52:04.772220 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1015dded-b387-4eb4-bd0d-1c833c86c12a admin admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:52:04.783937 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1015dded-b387-4eb4-bd0d-1c833c86c12a admin admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:52:04.856721 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baec3475-392c-43b7-92d8-fd833197ddbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:52:04.857392 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baec3475-392c-43b7-92d8-fd833197ddbe admin admin] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:52:04.857672 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baec3475-392c-43b7-92d8-fd833197ddbe admin admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:52:04.857945 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baec3475-392c-43b7-92d8-fd833197ddbe admin admin] PATH_INFO: `/v3/users` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:52:04.858674 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baec3475-392c-43b7-92d8-fd833197ddbe admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:52:04.860613 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baec3475-392c-43b7-92d8-fd833197ddbe admin admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:52:04.881537 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1389/2762] 10.222.0.57 () {60 vars in 1143 bytes} [Fri Jul 1 16:52:04 2022] POST /identity/v3/users => generated 303 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 16:52:04.886815 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9df1341-2797-4ab3-9963-81e6d680a3ce admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:52:04.897001 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9df1341-2797-4ab3-9963-81e6d680a3ce admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:52:04.902173 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e50b60-59d0-47d1-8cb0-67d19cd84cb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:52:04.902868 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e50b60-59d0-47d1-8cb0-67d19cd84cb0 admin admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:52:04.903154 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e50b60-59d0-47d1-8cb0-67d19cd84cb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:52:04.903428 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e50b60-59d0-47d1-8cb0-67d19cd84cb0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:52:04.904150 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e50b60-59d0-47d1-8cb0-67d19cd84cb0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:52:04.905743 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e50b60-59d0-47d1-8cb0-67d19cd84cb0 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:52:04.907845 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:52:04.917650 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1374/2763] 10.222.0.57 () {58 vars in 1121 bytes} [Fri Jul 1 16:52:04 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 16:52:04.922079 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baec3475-392c-43b7-92d8-fd833197ddbe admin admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:52:04.933198 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baec3475-392c-43b7-92d8-fd833197ddbe admin admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:52:04.937897 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d28779-ca05-4ac3-942c-6a1a911443d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:52:04.938590 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d28779-ca05-4ac3-942c-6a1a911443d0 admin admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:52:04.938876 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d28779-ca05-4ac3-942c-6a1a911443d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:52:04.939155 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d28779-ca05-4ac3-942c-6a1a911443d0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:52:04.939891 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d28779-ca05-4ac3-942c-6a1a911443d0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:52:04.941524 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d28779-ca05-4ac3-942c-6a1a911443d0 admin admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:52:04.952491 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1390/2764] 10.222.0.57 () {58 vars in 1121 bytes} [Fri Jul 1 16:52:04 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 16:52:04.956948 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e50b60-59d0-47d1-8cb0-67d19cd84cb0 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:52:04.967112 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e50b60-59d0-47d1-8cb0-67d19cd84cb0 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:52:04.971721 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b4e72d-be4f-4424-bfbc-8b94aa669006 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:52:04.972594 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b4e72d-be4f-4424-bfbc-8b94aa669006 admin admin] REQUEST_METHOD: `PUT` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:52:04.972882 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b4e72d-be4f-4424-bfbc-8b94aa669006 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:52:04.973160 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b4e72d-be4f-4424-bfbc-8b94aa669006 admin admin] PATH_INFO: `/v3/projects/72db4b3a23bd45c4a2c36bc42f5849e8/users/4183d531f3a74450a43a02b0564dd500/roles/983be40bdab04c478eaa90c6b6a46b14` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:52:04.988049 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b4e72d-be4f-4424-bfbc-8b94aa669006 admin admin] RBAC: Authorizing `identity:create_grant(project_id=72db4b3a23bd45c4a2c36bc42f5849e8, user_id=4183d531f3a74450a43a02b0564dd500, role_id=983be40bdab04c478eaa90c6b6a46b14)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:52:04.990789 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b4e72d-be4f-4424-bfbc-8b94aa669006 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:52:05.005622 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1375/2765] 10.222.0.57 () {60 vars in 1482 bytes} [Fri Jul 1 16:52:04 2022] PUT /identity/v3/projects/72db4b3a23bd45c4a2c36bc42f5849e8/users/4183d531f3a74450a43a02b0564dd500/roles/983be40bdab04c478eaa90c6b6a46b14 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 16:52:05.016401 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56feb3bc-9029-4bab-8e6a-e6d665aa7ed3 None None] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:52:05.016720 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56feb3bc-9029-4bab-8e6a-e6d665aa7ed3 None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:52:05.017054 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56feb3bc-9029-4bab-8e6a-e6d665aa7ed3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:52:05.043882 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.auth.core [None req-56feb3bc-9029-4bab-8e6a-e6d665aa7ed3 None None] MFA Rules not processed for user `4183d531f3a74450a43a02b0564dd500`. Rule list: `[]` (Enabled: `True`). {{(pid=80312) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:52:05.045169 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-56feb3bc-9029-4bab-8e6a-e6d665aa7ed3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:52:05.107724 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:52:05.112607 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1391/2766] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 1 16:52:05 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 16:52:05.117481 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b4e72d-be4f-4424-bfbc-8b94aa669006 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:52:05.192377 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b4e72d-be4f-4424-bfbc-8b94aa669006 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:52:05.198880 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fad3941-2eb4-4c59-af38-5d1112af876b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:52:05.199539 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fad3941-2eb4-4c59-af38-5d1112af876b admin admin] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:52:05.199820 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fad3941-2eb4-4c59-af38-5d1112af876b admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:52:05.200094 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fad3941-2eb4-4c59-af38-5d1112af876b admin admin] PATH_INFO: `/v3/projects` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:52:05.201953 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fad3941-2eb4-4c59-af38-5d1112af876b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:52:05.203892 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fad3941-2eb4-4c59-af38-5d1112af876b admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:52:05.230741 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1376/2767] 10.222.0.57 () {60 vars in 1152 bytes} [Fri Jul 1 16:52:05 2022] POST /identity/v3/projects => generated 403 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 16:52:05.234874 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56feb3bc-9029-4bab-8e6a-e6d665aa7ed3 None None] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:52:05.245780 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56feb3bc-9029-4bab-8e6a-e6d665aa7ed3 None None] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:52:05.301637 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5bd9ef7-c9e1-4c8c-aa73-624b16103480 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:52:05.302380 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bd9ef7-c9e1-4c8c-aa73-624b16103480 admin admin] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:52:05.302717 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bd9ef7-c9e1-4c8c-aa73-624b16103480 admin admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:52:05.302998 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bd9ef7-c9e1-4c8c-aa73-624b16103480 admin admin] PATH_INFO: `/v3/users` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:52:05.304205 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5bd9ef7-c9e1-4c8c-aa73-624b16103480 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:52:05.305814 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5bd9ef7-c9e1-4c8c-aa73-624b16103480 admin admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:52:05.326644 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1392/2768] 10.222.0.57 () {60 vars in 1143 bytes} [Fri Jul 1 16:52:05 2022] POST /identity/v3/users => generated 308 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 16:52:05.330571 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fad3941-2eb4-4c59-af38-5d1112af876b admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:52:05.341673 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fad3941-2eb4-4c59-af38-5d1112af876b admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:52:05.346265 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103b2979-e9e8-4128-9271-162b1022287f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:52:05.346946 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103b2979-e9e8-4128-9271-162b1022287f admin admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:52:05.347232 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103b2979-e9e8-4128-9271-162b1022287f admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:52:05.347510 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103b2979-e9e8-4128-9271-162b1022287f admin admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:52:05.348236 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-103b2979-e9e8-4128-9271-162b1022287f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:52:05.349851 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-103b2979-e9e8-4128-9271-162b1022287f admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:52:05.360557 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1377/2769] 10.222.0.57 () {58 vars in 1121 bytes} [Fri Jul 1 16:52:05 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 16:52:05.365417 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5bd9ef7-c9e1-4c8c-aa73-624b16103480 admin admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:52:05.375982 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5bd9ef7-c9e1-4c8c-aa73-624b16103480 admin admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:52:05.380639 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9c174e-56eb-4ad8-ad01-172482c53fb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:52:05.381514 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9c174e-56eb-4ad8-ad01-172482c53fb6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:52:05.382604 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9c174e-56eb-4ad8-ad01-172482c53fb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:52:05.382889 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9c174e-56eb-4ad8-ad01-172482c53fb6 admin admin] PATH_INFO: `/v3/projects/8451b91f37a14b5e89b7279a6d019e07/users/5fbef526cd504a4e8d7d52be7af86d45/roles/84ee739146934c1ab32c501ff8093644` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:52:05.398001 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc9c174e-56eb-4ad8-ad01-172482c53fb6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8451b91f37a14b5e89b7279a6d019e07, user_id=5fbef526cd504a4e8d7d52be7af86d45, role_id=84ee739146934c1ab32c501ff8093644)` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:52:05.400652 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc9c174e-56eb-4ad8-ad01-172482c53fb6 admin admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:52:05.413886 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1393/2770] 10.222.0.57 () {60 vars in 1482 bytes} [Fri Jul 1 16:52:05 2022] PUT /identity/v3/projects/8451b91f37a14b5e89b7279a6d019e07/users/5fbef526cd504a4e8d7d52be7af86d45/roles/84ee739146934c1ab32c501ff8093644 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 16:52:05.417959 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103b2979-e9e8-4128-9271-162b1022287f admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:52:05.428194 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103b2979-e9e8-4128-9271-162b1022287f admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:52:05.488706 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87eeb62e-6711-4e42-b5b6-950aa2e85239 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:52:05.489293 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87eeb62e-6711-4e42-b5b6-950aa2e85239 admin admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:52:05.489575 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87eeb62e-6711-4e42-b5b6-950aa2e85239 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:52:05.489859 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87eeb62e-6711-4e42-b5b6-950aa2e85239 admin admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:52:05.490622 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87eeb62e-6711-4e42-b5b6-950aa2e85239 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:52:05.492843 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87eeb62e-6711-4e42-b5b6-950aa2e85239 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:52:05.503359 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1378/2771] 10.222.0.57 () {58 vars in 1121 bytes} [Fri Jul 1 16:52:05 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 16:52:05.508352 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9c174e-56eb-4ad8-ad01-172482c53fb6 admin admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:52:05.519092 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9c174e-56eb-4ad8-ad01-172482c53fb6 admin admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:52:05.523921 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc332eb4-bb1a-46c2-a3cd-bffe23197a4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:52:05.524816 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc332eb4-bb1a-46c2-a3cd-bffe23197a4d admin admin] REQUEST_METHOD: `PUT` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:52:05.525103 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc332eb4-bb1a-46c2-a3cd-bffe23197a4d admin admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:52:05.525383 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc332eb4-bb1a-46c2-a3cd-bffe23197a4d admin admin] PATH_INFO: `/v3/projects/8451b91f37a14b5e89b7279a6d019e07/users/5fbef526cd504a4e8d7d52be7af86d45/roles/84ee739146934c1ab32c501ff8093644` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:52:05.527381 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc332eb4-bb1a-46c2-a3cd-bffe23197a4d admin admin] RBAC: Authorizing `identity:create_grant(project_id=8451b91f37a14b5e89b7279a6d019e07, user_id=5fbef526cd504a4e8d7d52be7af86d45, role_id=84ee739146934c1ab32c501ff8093644)` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:52:05.530050 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc332eb4-bb1a-46c2-a3cd-bffe23197a4d admin admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:52:05.541200 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1394/2772] 10.222.0.57 () {60 vars in 1482 bytes} [Fri Jul 1 16:52:05 2022] PUT /identity/v3/projects/8451b91f37a14b5e89b7279a6d019e07/users/5fbef526cd504a4e8d7d52be7af86d45/roles/84ee739146934c1ab32c501ff8093644 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 16:52:05.552537 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8773ec52-e0e3-4c59-82e2-c9896679697b None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:52:05.552823 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8773ec52-e0e3-4c59-82e2-c9896679697b None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:52:05.553232 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8773ec52-e0e3-4c59-82e2-c9896679697b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:52:05.581695 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-8773ec52-e0e3-4c59-82e2-c9896679697b None None] MFA Rules not processed for user `5fbef526cd504a4e8d7d52be7af86d45`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:52:05.582963 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-8773ec52-e0e3-4c59-82e2-c9896679697b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:52:05.615882 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:52:05.654510 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1379/2773] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 1 16:52:05 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 16:52:05.666592 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc332eb4-bb1a-46c2-a3cd-bffe23197a4d admin admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:52:05.668328 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-cc332eb4-bb1a-46c2-a3cd-bffe23197a4d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:52:05.727185 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc332eb4-bb1a-46c2-a3cd-bffe23197a4d admin admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:52:05.730883 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c40430-727c-405d-a792-6f453220e762 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbddf971a2354bbdbb9da1d63f3abf88', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0788e6725a84e57a4174284c1845755', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:52:05.731529 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c40430-727c-405d-a792-6f453220e762 service cinder] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:52:05.731815 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c40430-727c-405d-a792-6f453220e762 service cinder] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:52:05.732092 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c40430-727c-405d-a792-6f453220e762 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:52:05.741592 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c40430-727c-405d-a792-6f453220e762 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:52:05.743042 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c40430-727c-405d-a792-6f453220e762 service cinder] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:52:05.752368 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1395/2774] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 1 16:52:05 2022] GET /identity/v3/auth/tokens => generated 3337 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 16:52:05.911387 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8773ec52-e0e3-4c59-82e2-c9896679697b None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:52:05.972414 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8773ec52-e0e3-4c59-82e2-c9896679697b None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:52:05.977517 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af03ff5-0402-48bb-aab2-e9ecc81713f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:52:05.978402 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af03ff5-0402-48bb-aab2-e9ecc81713f7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:52:05.978687 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af03ff5-0402-48bb-aab2-e9ecc81713f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:52:05.978963 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af03ff5-0402-48bb-aab2-e9ecc81713f7 admin admin] PATH_INFO: `/v3/users/4183d531f3a74450a43a02b0564dd500` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:52:05.980214 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af03ff5-0402-48bb-aab2-e9ecc81713f7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4183d531f3a74450a43a02b0564dd500)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:52:05.982121 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af03ff5-0402-48bb-aab2-e9ecc81713f7 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:52:06.027784 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-2af03ff5-0402-48bb-aab2-e9ecc81713f7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4183d531f3a74450a43a02b0564dd500'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:52:06.028709 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-2af03ff5-0402-48bb-aab2-e9ecc81713f7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4183d531f3a74450a43a02b0564dd500'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:52:06.029328 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-2af03ff5-0402-48bb-aab2-e9ecc81713f7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4183d531f3a74450a43a02b0564dd500'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:52:06.036603 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-2af03ff5-0402-48bb-aab2-e9ecc81713f7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4183d531f3a74450a43a02b0564dd500'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:52:06.049504 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-2af03ff5-0402-48bb-aab2-e9ecc81713f7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4183d531f3a74450a43a02b0564dd500'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:52:06.062800 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1380/2775] 10.222.0.57 () {58 vars in 1223 bytes} [Fri Jul 1 16:52:05 2022] DELETE /identity/v3/users/4183d531f3a74450a43a02b0564dd500 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 01 16:52:06.067430 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17e7cc3-c3a6-4cdd-a68c-87a6ab6bf03e None None] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:52:06.067912 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17e7cc3-c3a6-4cdd-a68c-87a6ab6bf03e None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:52:06.068013 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17e7cc3-c3a6-4cdd-a68c-87a6ab6bf03e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:52:06.085903 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.auth.core [None req-e17e7cc3-c3a6-4cdd-a68c-87a6ab6bf03e None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80312) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:52:06.087125 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-e17e7cc3-c3a6-4cdd-a68c-87a6ab6bf03e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:52:06.134942 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:52:06.147126 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1396/2776] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 1 16:52:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 16:52:06.156914 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af03ff5-0402-48bb-aab2-e9ecc81713f7 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:52:06.158368 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-2af03ff5-0402-48bb-aab2-e9ecc81713f7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:52:06.213421 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:52:06.217666 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af03ff5-0402-48bb-aab2-e9ecc81713f7 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:52:06.222467 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26acaea7-1299-4bb0-bfa9-73737e688413 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ed200ce29684c5f80354fa9a8638fe2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0788e6725a84e57a4174284c1845755', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:52:06.223413 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26acaea7-1299-4bb0-bfa9-73737e688413 service neutron] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:52:06.223874 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26acaea7-1299-4bb0-bfa9-73737e688413 service neutron] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:52:06.224337 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26acaea7-1299-4bb0-bfa9-73737e688413 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:52:06.234312 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26acaea7-1299-4bb0-bfa9-73737e688413 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:52:06.235772 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26acaea7-1299-4bb0-bfa9-73737e688413 service neutron] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:52:06.245305 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1381/2777] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 1 16:52:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 16:52:06.519251 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17e7cc3-c3a6-4cdd-a68c-87a6ab6bf03e None None] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:52:06.520733 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-e17e7cc3-c3a6-4cdd-a68c-87a6ab6bf03e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:52:06.537080 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17e7cc3-c3a6-4cdd-a68c-87a6ab6bf03e None None] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:52:06.542869 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411312e1-eed8-4bac-86c8-23a8badc0c84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:52:06.543740 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411312e1-eed8-4bac-86c8-23a8badc0c84 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:52:06.544130 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411312e1-eed8-4bac-86c8-23a8badc0c84 admin admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:52:06.544301 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411312e1-eed8-4bac-86c8-23a8badc0c84 admin admin] PATH_INFO: `/v3/projects/72db4b3a23bd45c4a2c36bc42f5849e8` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:52:06.545532 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411312e1-eed8-4bac-86c8-23a8badc0c84 admin admin] RBAC: Authorizing `identity:delete_project(project_id=72db4b3a23bd45c4a2c36bc42f5849e8)` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:52:06.547484 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411312e1-eed8-4bac-86c8-23a8badc0c84 admin admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:52:06.594091 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-411312e1-eed8-4bac-86c8-23a8badc0c84 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '72db4b3a23bd45c4a2c36bc42f5849e8', 'request_id': 'req-411312e1-eed8-4bac-86c8-23a8badc0c84'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:52:06.603915 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-411312e1-eed8-4bac-86c8-23a8badc0c84 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '72db4b3a23bd45c4a2c36bc42f5849e8', 'request_id': 'req-411312e1-eed8-4bac-86c8-23a8badc0c84'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:52:06.609876 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-411312e1-eed8-4bac-86c8-23a8badc0c84 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '72db4b3a23bd45c4a2c36bc42f5849e8', 'request_id': 'req-411312e1-eed8-4bac-86c8-23a8badc0c84'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:52:06.621385 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-411312e1-eed8-4bac-86c8-23a8badc0c84 admin admin] The token cache is being invalidate because project 72db4b3a23bd45c4a2c36bc42f5849e8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80312) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 01 16:52:06.621674 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-411312e1-eed8-4bac-86c8-23a8badc0c84 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:52:06.622404 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-411312e1-eed8-4bac-86c8-23a8badc0c84 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:52:06.623962 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1397/2778] 10.222.0.57 () {58 vars in 1232 bytes} [Fri Jul 1 16:52:06 2022] DELETE /identity/v3/projects/72db4b3a23bd45c4a2c36bc42f5849e8 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 16:52:06.627928 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26acaea7-1299-4bb0-bfa9-73737e688413 service neutron] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:52:06.638791 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26acaea7-1299-4bb0-bfa9-73737e688413 service neutron] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:52:06.698107 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e95648c-3bcf-4f40-a1b6-cdf66617f82f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:52:06.699003 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e95648c-3bcf-4f40-a1b6-cdf66617f82f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:52:06.699291 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e95648c-3bcf-4f40-a1b6-cdf66617f82f admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:52:06.699572 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e95648c-3bcf-4f40-a1b6-cdf66617f82f admin admin] PATH_INFO: `/v3/users/5fbef526cd504a4e8d7d52be7af86d45` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:52:06.700758 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e95648c-3bcf-4f40-a1b6-cdf66617f82f admin admin] RBAC: Authorizing `identity:delete_user(user_id=5fbef526cd504a4e8d7d52be7af86d45)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:52:06.702715 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e95648c-3bcf-4f40-a1b6-cdf66617f82f admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:52:06.749022 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-4e95648c-3bcf-4f40-a1b6-cdf66617f82f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5fbef526cd504a4e8d7d52be7af86d45'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:52:06.749683 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-4e95648c-3bcf-4f40-a1b6-cdf66617f82f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5fbef526cd504a4e8d7d52be7af86d45'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:52:06.750291 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-4e95648c-3bcf-4f40-a1b6-cdf66617f82f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5fbef526cd504a4e8d7d52be7af86d45'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:52:06.758223 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-4e95648c-3bcf-4f40-a1b6-cdf66617f82f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5fbef526cd504a4e8d7d52be7af86d45'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:52:06.771810 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-4e95648c-3bcf-4f40-a1b6-cdf66617f82f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5fbef526cd504a4e8d7d52be7af86d45'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:52:06.786111 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1382/2779] 10.222.0.57 () {58 vars in 1223 bytes} [Fri Jul 1 16:52:06 2022] DELETE /identity/v3/users/5fbef526cd504a4e8d7d52be7af86d45 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 16:52:07.070794 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411312e1-eed8-4bac-86c8-23a8badc0c84 admin admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:52:07.072321 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-411312e1-eed8-4bac-86c8-23a8badc0c84 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:52:07.087441 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411312e1-eed8-4bac-86c8-23a8badc0c84 admin admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:52:07.156014 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2001679a-9fe1-4f7c-b135-fdcfecbce0a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:52:07.156861 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2001679a-9fe1-4f7c-b135-fdcfecbce0a9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:52:07.157161 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2001679a-9fe1-4f7c-b135-fdcfecbce0a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:52:07.157418 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2001679a-9fe1-4f7c-b135-fdcfecbce0a9 admin admin] PATH_INFO: `/v3/projects/8451b91f37a14b5e89b7279a6d019e07` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:52:07.158652 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2001679a-9fe1-4f7c-b135-fdcfecbce0a9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8451b91f37a14b5e89b7279a6d019e07)` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:52:07.160586 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2001679a-9fe1-4f7c-b135-fdcfecbce0a9 admin admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:52:07.207450 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-2001679a-9fe1-4f7c-b135-fdcfecbce0a9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8451b91f37a14b5e89b7279a6d019e07', 'request_id': 'req-2001679a-9fe1-4f7c-b135-fdcfecbce0a9'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:52:07.215257 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-2001679a-9fe1-4f7c-b135-fdcfecbce0a9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8451b91f37a14b5e89b7279a6d019e07', 'request_id': 'req-2001679a-9fe1-4f7c-b135-fdcfecbce0a9'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:52:07.221083 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-2001679a-9fe1-4f7c-b135-fdcfecbce0a9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8451b91f37a14b5e89b7279a6d019e07', 'request_id': 'req-2001679a-9fe1-4f7c-b135-fdcfecbce0a9'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:52:07.232884 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-2001679a-9fe1-4f7c-b135-fdcfecbce0a9 admin admin] The token cache is being invalidate because project 8451b91f37a14b5e89b7279a6d019e07 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80312) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 01 16:52:07.233175 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-2001679a-9fe1-4f7c-b135-fdcfecbce0a9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:52:07.233824 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-2001679a-9fe1-4f7c-b135-fdcfecbce0a9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:52:07.235501 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1398/2780] 10.222.0.57 () {58 vars in 1232 bytes} [Fri Jul 1 16:52:07 2022] DELETE /identity/v3/projects/8451b91f37a14b5e89b7279a6d019e07 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 16:52:07.240529 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e0f0d6-8afe-4e3a-8bb4-8c5526ef8858 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:52:07.240847 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e0f0d6-8afe-4e3a-8bb4-8c5526ef8858 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:52:07.241262 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e0f0d6-8afe-4e3a-8bb4-8c5526ef8858 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:52:07.259337 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-19e0f0d6-8afe-4e3a-8bb4-8c5526ef8858 None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:52:07.260519 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-19e0f0d6-8afe-4e3a-8bb4-8c5526ef8858 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:52:07.319295 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1383/2781] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 1 16:52:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 16:52:07.327463 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57fe200-5303-4ab1-9b2f-40e036881d03 None None] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:52:07.327761 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57fe200-5303-4ab1-9b2f-40e036881d03 None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:52:07.328046 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57fe200-5303-4ab1-9b2f-40e036881d03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:52:07.347465 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.auth.core [None req-d57fe200-5303-4ab1-9b2f-40e036881d03 None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80312) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:52:07.348684 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-d57fe200-5303-4ab1-9b2f-40e036881d03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:52:07.357916 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1399/2782] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 1 16:52:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 16:52:07.362077 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e0f0d6-8afe-4e3a-8bb4-8c5526ef8858 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:52:07.371932 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e0f0d6-8afe-4e3a-8bb4-8c5526ef8858 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:52:07.376555 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018a032f-950c-460b-a097-7dfc2ebc1ec8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:52:07.377243 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018a032f-950c-460b-a097-7dfc2ebc1ec8 admin admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:52:07.377534 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018a032f-950c-460b-a097-7dfc2ebc1ec8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:52:07.377814 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018a032f-950c-460b-a097-7dfc2ebc1ec8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:52:07.378620 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-018a032f-950c-460b-a097-7dfc2ebc1ec8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:52:07.380261 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-018a032f-950c-460b-a097-7dfc2ebc1ec8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:52:07.380489 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-018a032f-950c-460b-a097-7dfc2ebc1ec8 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:52:07.390914 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:52:07.397922 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1384/2783] 10.222.0.57 () {58 vars in 1152 bytes} [Fri Jul 1 16:52:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:52:07.402732 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57fe200-5303-4ab1-9b2f-40e036881d03 None None] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:52:07.407591 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:52:07.414421 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57fe200-5303-4ab1-9b2f-40e036881d03 None None] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:52:07.419995 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d694e7c3-ea13-4150-a162-28b66e2ff7b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:52:07.420681 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d694e7c3-ea13-4150-a162-28b66e2ff7b1 admin admin] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:52:07.420963 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d694e7c3-ea13-4150-a162-28b66e2ff7b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:52:07.421236 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d694e7c3-ea13-4150-a162-28b66e2ff7b1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:52:07.421921 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d694e7c3-ea13-4150-a162-28b66e2ff7b1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:52:07.423916 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d694e7c3-ea13-4150-a162-28b66e2ff7b1 admin admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:52:07.449609 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1400/2784] 10.222.0.57 () {60 vars in 1152 bytes} [Fri Jul 1 16:52:07 2022] POST /identity/v3/projects => generated 375 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 01 16:52:07.453767 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018a032f-950c-460b-a097-7dfc2ebc1ec8 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:52:07.465067 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018a032f-950c-460b-a097-7dfc2ebc1ec8 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:52:07.523088 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41da9a1c-7394-40f6-9035-7178313b3648 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:52:07.523737 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41da9a1c-7394-40f6-9035-7178313b3648 admin admin] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:52:07.524026 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41da9a1c-7394-40f6-9035-7178313b3648 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:52:07.524305 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41da9a1c-7394-40f6-9035-7178313b3648 admin admin] PATH_INFO: `/v3/users` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:52:07.525013 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41da9a1c-7394-40f6-9035-7178313b3648 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:52:07.526989 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41da9a1c-7394-40f6-9035-7178313b3648 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:52:07.547362 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1385/2785] 10.222.0.57 () {60 vars in 1143 bytes} [Fri Jul 1 16:52:07 2022] POST /identity/v3/users => generated 288 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 01 16:52:07.551400 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d694e7c3-ea13-4150-a162-28b66e2ff7b1 admin admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:52:07.562671 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d694e7c3-ea13-4150-a162-28b66e2ff7b1 admin admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:52:07.567362 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ba858a-dce3-474b-a2d2-52e6b913c1bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:52:07.568022 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ba858a-dce3-474b-a2d2-52e6b913c1bf admin admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:52:07.568311 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ba858a-dce3-474b-a2d2-52e6b913c1bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:52:07.568593 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ba858a-dce3-474b-a2d2-52e6b913c1bf admin admin] PATH_INFO: `/v3/roles` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:52:07.570304 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ba858a-dce3-474b-a2d2-52e6b913c1bf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:52:07.571737 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ba858a-dce3-474b-a2d2-52e6b913c1bf admin admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:52:07.582548 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1401/2786] 10.222.0.57 () {58 vars in 1121 bytes} [Fri Jul 1 16:52:07 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 01 16:52:07.587339 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41da9a1c-7394-40f6-9035-7178313b3648 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:52:07.597222 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41da9a1c-7394-40f6-9035-7178313b3648 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:52:07.602063 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16c95f1-46f2-480a-9c81-1f86782b6923 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:52:07.602684 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16c95f1-46f2-480a-9c81-1f86782b6923 admin admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:52:07.603055 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16c95f1-46f2-480a-9c81-1f86782b6923 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:52:07.603398 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16c95f1-46f2-480a-9c81-1f86782b6923 admin admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:52:07.604124 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e16c95f1-46f2-480a-9c81-1f86782b6923 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:52:07.605738 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e16c95f1-46f2-480a-9c81-1f86782b6923 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:52:07.615861 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1386/2787] 10.222.0.57 () {58 vars in 1121 bytes} [Fri Jul 1 16:52:07 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 16:52:07.619877 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ba858a-dce3-474b-a2d2-52e6b913c1bf admin admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:52:07.629681 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ba858a-dce3-474b-a2d2-52e6b913c1bf admin admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:52:07.634830 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d30c27-d788-40bc-918d-1561db4b2b63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:52:07.635704 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d30c27-d788-40bc-918d-1561db4b2b63 admin admin] REQUEST_METHOD: `PUT` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:52:07.635991 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d30c27-d788-40bc-918d-1561db4b2b63 admin admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:52:07.636272 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d30c27-d788-40bc-918d-1561db4b2b63 admin admin] PATH_INFO: `/v3/projects/9cc592f9f66a4ffc84a4550586e9a0f4/users/4a88502ab66949a4bc5e83665d4421ce/roles/983be40bdab04c478eaa90c6b6a46b14` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:52:07.651323 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d30c27-d788-40bc-918d-1561db4b2b63 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9cc592f9f66a4ffc84a4550586e9a0f4, user_id=4a88502ab66949a4bc5e83665d4421ce, role_id=983be40bdab04c478eaa90c6b6a46b14)` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:52:07.653971 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d30c27-d788-40bc-918d-1561db4b2b63 admin admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:52:07.667926 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1402/2788] 10.222.0.57 () {60 vars in 1482 bytes} [Fri Jul 1 16:52:07 2022] PUT /identity/v3/projects/9cc592f9f66a4ffc84a4550586e9a0f4/users/4a88502ab66949a4bc5e83665d4421ce/roles/983be40bdab04c478eaa90c6b6a46b14 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 16:52:07.679434 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d70bceb-4cbf-4676-91d6-58db5335a8c4 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:52:07.679914 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d70bceb-4cbf-4676-91d6-58db5335a8c4 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:52:07.680032 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d70bceb-4cbf-4676-91d6-58db5335a8c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:52:07.708377 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-3d70bceb-4cbf-4676-91d6-58db5335a8c4 None None] MFA Rules not processed for user `4a88502ab66949a4bc5e83665d4421ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:52:07.709595 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-3d70bceb-4cbf-4676-91d6-58db5335a8c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:52:07.773273 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1387/2789] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 1 16:52:07 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 16:52:07.777828 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d30c27-d788-40bc-918d-1561db4b2b63 admin admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:52:07.840762 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d30c27-d788-40bc-918d-1561db4b2b63 admin admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:52:07.845516 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66be4f33-ce00-43d3-81d5-56f01284582c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:52:07.846179 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66be4f33-ce00-43d3-81d5-56f01284582c admin admin] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:52:07.846492 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66be4f33-ce00-43d3-81d5-56f01284582c admin admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:52:07.846771 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66be4f33-ce00-43d3-81d5-56f01284582c admin admin] PATH_INFO: `/v3/projects` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:52:07.847482 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66be4f33-ce00-43d3-81d5-56f01284582c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:52:07.849389 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66be4f33-ce00-43d3-81d5-56f01284582c admin admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:52:07.875101 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1403/2790] 10.222.0.57 () {60 vars in 1152 bytes} [Fri Jul 1 16:52:07 2022] POST /identity/v3/projects => generated 377 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 16:52:07.879205 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d70bceb-4cbf-4676-91d6-58db5335a8c4 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:52:07.890387 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d70bceb-4cbf-4676-91d6-58db5335a8c4 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:52:07.945820 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0152a4-ada8-4285-aa64-9344b5528df3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:52:07.946516 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0152a4-ada8-4285-aa64-9344b5528df3 admin admin] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:52:07.946806 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0152a4-ada8-4285-aa64-9344b5528df3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:52:07.947088 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0152a4-ada8-4285-aa64-9344b5528df3 admin admin] PATH_INFO: `/v3/users` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:52:07.947778 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f0152a4-ada8-4285-aa64-9344b5528df3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:52:07.949705 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f0152a4-ada8-4285-aa64-9344b5528df3 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:52:07.959296 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:52:07.970200 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1388/2791] 10.222.0.57 () {60 vars in 1143 bytes} [Fri Jul 1 16:52:07 2022] POST /identity/v3/users => generated 295 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 16:52:07.974201 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66be4f33-ce00-43d3-81d5-56f01284582c admin admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:52:07.985173 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66be4f33-ce00-43d3-81d5-56f01284582c admin admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:52:07.989729 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c27e4a8-25b3-4f81-b05c-66ce063f0e36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:52:07.990415 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c27e4a8-25b3-4f81-b05c-66ce063f0e36 admin admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:52:07.990705 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c27e4a8-25b3-4f81-b05c-66ce063f0e36 admin admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:52:07.991552 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c27e4a8-25b3-4f81-b05c-66ce063f0e36 admin admin] PATH_INFO: `/v3/roles` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:52:07.992829 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c27e4a8-25b3-4f81-b05c-66ce063f0e36 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:52:07.994506 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c27e4a8-25b3-4f81-b05c-66ce063f0e36 admin admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:52:08.005103 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1404/2792] 10.222.0.57 () {58 vars in 1121 bytes} [Fri Jul 1 16:52:07 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 01 16:52:08.009953 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0152a4-ada8-4285-aa64-9344b5528df3 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:52:08.019917 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0152a4-ada8-4285-aa64-9344b5528df3 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:52:08.026704 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2f9f71-31e9-4c28-b318-5f85652309ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:52:08.027631 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2f9f71-31e9-4c28-b318-5f85652309ee admin admin] REQUEST_METHOD: `PUT` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:52:08.027964 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2f9f71-31e9-4c28-b318-5f85652309ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:52:08.028286 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2f9f71-31e9-4c28-b318-5f85652309ee admin admin] PATH_INFO: `/v3/projects/81754900e29445e88205dbc82c8d4145/users/e2bb569673634c749168d8e443b8a07b/roles/84ee739146934c1ab32c501ff8093644` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:52:08.046855 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce2f9f71-31e9-4c28-b318-5f85652309ee admin admin] RBAC: Authorizing `identity:create_grant(project_id=81754900e29445e88205dbc82c8d4145, user_id=e2bb569673634c749168d8e443b8a07b, role_id=84ee739146934c1ab32c501ff8093644)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:52:08.049562 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce2f9f71-31e9-4c28-b318-5f85652309ee admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:52:08.063027 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1389/2793] 10.222.0.57 () {60 vars in 1482 bytes} [Fri Jul 1 16:52:08 2022] PUT /identity/v3/projects/81754900e29445e88205dbc82c8d4145/users/e2bb569673634c749168d8e443b8a07b/roles/84ee739146934c1ab32c501ff8093644 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 01 16:52:08.067168 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c27e4a8-25b3-4f81-b05c-66ce063f0e36 admin admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:52:08.077222 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c27e4a8-25b3-4f81-b05c-66ce063f0e36 admin admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:52:08.135766 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f881701a-ab40-429b-bc99-c630253af491 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:52:08.136350 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f881701a-ab40-429b-bc99-c630253af491 admin admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:52:08.136643 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f881701a-ab40-429b-bc99-c630253af491 admin admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:52:08.136927 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f881701a-ab40-429b-bc99-c630253af491 admin admin] PATH_INFO: `/v3/roles` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:52:08.137660 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f881701a-ab40-429b-bc99-c630253af491 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:52:08.139297 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f881701a-ab40-429b-bc99-c630253af491 admin admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:52:08.150492 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1405/2794] 10.222.0.57 () {58 vars in 1121 bytes} [Fri Jul 1 16:52:08 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 16:52:08.154920 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2f9f71-31e9-4c28-b318-5f85652309ee admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:52:08.165787 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2f9f71-31e9-4c28-b318-5f85652309ee admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:52:08.171231 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c737101c-b703-4eec-85b3-7a3a22490402 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:52:08.172397 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c737101c-b703-4eec-85b3-7a3a22490402 admin admin] REQUEST_METHOD: `PUT` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:52:08.172701 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c737101c-b703-4eec-85b3-7a3a22490402 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:52:08.173082 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c737101c-b703-4eec-85b3-7a3a22490402 admin admin] PATH_INFO: `/v3/projects/81754900e29445e88205dbc82c8d4145/users/e2bb569673634c749168d8e443b8a07b/roles/84ee739146934c1ab32c501ff8093644` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:52:08.175094 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c737101c-b703-4eec-85b3-7a3a22490402 admin admin] RBAC: Authorizing `identity:create_grant(project_id=81754900e29445e88205dbc82c8d4145, user_id=e2bb569673634c749168d8e443b8a07b, role_id=84ee739146934c1ab32c501ff8093644)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:52:08.177727 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c737101c-b703-4eec-85b3-7a3a22490402 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:52:08.188545 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1390/2795] 10.222.0.57 () {60 vars in 1482 bytes} [Fri Jul 1 16:52:08 2022] PUT /identity/v3/projects/81754900e29445e88205dbc82c8d4145/users/e2bb569673634c749168d8e443b8a07b/roles/84ee739146934c1ab32c501ff8093644 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 16:52:08.199467 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4a873b-8cf6-44c4-8bea-0b8361c26485 None None] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:52:08.199766 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4a873b-8cf6-44c4-8bea-0b8361c26485 None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:52:08.200050 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4a873b-8cf6-44c4-8bea-0b8361c26485 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:52:08.227915 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.auth.core [None req-ab4a873b-8cf6-44c4-8bea-0b8361c26485 None None] MFA Rules not processed for user `e2bb569673634c749168d8e443b8a07b`. Rule list: `[]` (Enabled: `True`). {{(pid=80312) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:52:08.229136 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-ab4a873b-8cf6-44c4-8bea-0b8361c26485 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:52:08.233077 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:52:08.299336 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1406/2796] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 1 16:52:08 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 16:52:08.310688 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c737101c-b703-4eec-85b3-7a3a22490402 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:52:08.312416 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-c737101c-b703-4eec-85b3-7a3a22490402 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:52:08.370486 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c737101c-b703-4eec-85b3-7a3a22490402 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:52:08.374130 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d27bfa-e465-4126-81a7-01081b735450 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbddf971a2354bbdbb9da1d63f3abf88', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0788e6725a84e57a4174284c1845755', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:52:08.374802 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d27bfa-e465-4126-81a7-01081b735450 service cinder] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:52:08.375089 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d27bfa-e465-4126-81a7-01081b735450 service cinder] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:52:08.375367 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d27bfa-e465-4126-81a7-01081b735450 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:52:08.432739 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d27bfa-e465-4126-81a7-01081b735450 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:52:08.434469 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d27bfa-e465-4126-81a7-01081b735450 service cinder] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:52:08.443515 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1391/2797] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 1 16:52:08 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 16:52:14.067801 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4a873b-8cf6-44c4-8bea-0b8361c26485 None None] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:52:14.069202 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-ab4a873b-8cf6-44c4-8bea-0b8361c26485 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:52:14.081172 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4a873b-8cf6-44c4-8bea-0b8361c26485 None None] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:52:14.085497 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181e6446-0a20-42ae-8eef-d5276c3812fc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbddf971a2354bbdbb9da1d63f3abf88', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0788e6725a84e57a4174284c1845755', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:52:14.086219 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181e6446-0a20-42ae-8eef-d5276c3812fc service cinder] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:52:14.086552 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181e6446-0a20-42ae-8eef-d5276c3812fc service cinder] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:52:14.086835 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181e6446-0a20-42ae-8eef-d5276c3812fc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:52:14.095869 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-181e6446-0a20-42ae-8eef-d5276c3812fc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:52:14.097297 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-181e6446-0a20-42ae-8eef-d5276c3812fc service cinder] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:52:14.107751 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1407/2798] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 1 16:52:14 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 16:52:18.439338 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=65,keystone:INSERT=2 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:52:24.103459 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=33 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:52:38.925211 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d27bfa-e465-4126-81a7-01081b735450 service cinder] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:52:38.987827 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d27bfa-e465-4126-81a7-01081b735450 service cinder] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:52:38.993350 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0563bb6c-b473-48ae-825c-1581369b6251 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:52:38.994221 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0563bb6c-b473-48ae-825c-1581369b6251 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:52:38.994541 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0563bb6c-b473-48ae-825c-1581369b6251 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:52:38.994823 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0563bb6c-b473-48ae-825c-1581369b6251 admin admin] PATH_INFO: `/v3/users/4a88502ab66949a4bc5e83665d4421ce` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:52:38.996246 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0563bb6c-b473-48ae-825c-1581369b6251 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4a88502ab66949a4bc5e83665d4421ce)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:52:38.998152 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0563bb6c-b473-48ae-825c-1581369b6251 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:52:39.045286 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-0563bb6c-b473-48ae-825c-1581369b6251 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4a88502ab66949a4bc5e83665d4421ce'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:52:39.045946 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-0563bb6c-b473-48ae-825c-1581369b6251 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4a88502ab66949a4bc5e83665d4421ce'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:52:39.046600 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-0563bb6c-b473-48ae-825c-1581369b6251 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4a88502ab66949a4bc5e83665d4421ce'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:52:39.055276 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-0563bb6c-b473-48ae-825c-1581369b6251 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4a88502ab66949a4bc5e83665d4421ce'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:52:39.067238 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-0563bb6c-b473-48ae-825c-1581369b6251 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4a88502ab66949a4bc5e83665d4421ce'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:52:39.079552 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1392/2799] 10.222.0.57 () {58 vars in 1223 bytes} [Fri Jul 1 16:52:38 2022] DELETE /identity/v3/users/4a88502ab66949a4bc5e83665d4421ce => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 16:52:39.084251 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf083b90-3ca8-4a67-aa7a-edad2b98dd1f None None] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:52:39.084727 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf083b90-3ca8-4a67-aa7a-edad2b98dd1f None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:52:39.085319 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf083b90-3ca8-4a67-aa7a-edad2b98dd1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:52:39.103450 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.auth.core [None req-cf083b90-3ca8-4a67-aa7a-edad2b98dd1f None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80312) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:52:39.104650 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-cf083b90-3ca8-4a67-aa7a-edad2b98dd1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:52:39.164329 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1408/2800] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 1 16:52:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 16:52:39.174026 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0563bb6c-b473-48ae-825c-1581369b6251 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:52:39.175479 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-0563bb6c-b473-48ae-825c-1581369b6251 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:52:39.237951 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0563bb6c-b473-48ae-825c-1581369b6251 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:52:39.242288 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bedadefb-57c1-440a-b59f-254d3aa37774 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ed200ce29684c5f80354fa9a8638fe2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0788e6725a84e57a4174284c1845755', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:52:39.242966 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedadefb-57c1-440a-b59f-254d3aa37774 service neutron] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:52:39.243254 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedadefb-57c1-440a-b59f-254d3aa37774 service neutron] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:52:39.243576 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedadefb-57c1-440a-b59f-254d3aa37774 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:52:39.252654 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bedadefb-57c1-440a-b59f-254d3aa37774 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:52:39.254084 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bedadefb-57c1-440a-b59f-254d3aa37774 service neutron] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:52:39.264400 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1393/2801] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 1 16:52:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 16:52:39.538844 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf083b90-3ca8-4a67-aa7a-edad2b98dd1f None None] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:52:39.540253 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-cf083b90-3ca8-4a67-aa7a-edad2b98dd1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:52:39.552812 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf083b90-3ca8-4a67-aa7a-edad2b98dd1f None None] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:52:39.559929 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55c6de6-38cc-4ada-b99f-f77efb9cdea7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:52:39.561189 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55c6de6-38cc-4ada-b99f-f77efb9cdea7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:52:39.561604 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55c6de6-38cc-4ada-b99f-f77efb9cdea7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:52:39.561961 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55c6de6-38cc-4ada-b99f-f77efb9cdea7 admin admin] PATH_INFO: `/v3/projects/9cc592f9f66a4ffc84a4550586e9a0f4` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:52:39.563547 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c55c6de6-38cc-4ada-b99f-f77efb9cdea7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9cc592f9f66a4ffc84a4550586e9a0f4)` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:52:39.565593 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c55c6de6-38cc-4ada-b99f-f77efb9cdea7 admin admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:52:39.628619 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-c55c6de6-38cc-4ada-b99f-f77efb9cdea7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9cc592f9f66a4ffc84a4550586e9a0f4', 'request_id': 'req-c55c6de6-38cc-4ada-b99f-f77efb9cdea7'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:52:39.639589 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-c55c6de6-38cc-4ada-b99f-f77efb9cdea7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9cc592f9f66a4ffc84a4550586e9a0f4', 'request_id': 'req-c55c6de6-38cc-4ada-b99f-f77efb9cdea7'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:52:39.645752 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-c55c6de6-38cc-4ada-b99f-f77efb9cdea7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9cc592f9f66a4ffc84a4550586e9a0f4', 'request_id': 'req-c55c6de6-38cc-4ada-b99f-f77efb9cdea7'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:52:39.658227 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-c55c6de6-38cc-4ada-b99f-f77efb9cdea7 admin admin] The token cache is being invalidate because project 9cc592f9f66a4ffc84a4550586e9a0f4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80312) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 01 16:52:39.658614 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-c55c6de6-38cc-4ada-b99f-f77efb9cdea7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:52:39.659252 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-c55c6de6-38cc-4ada-b99f-f77efb9cdea7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:52:39.661285 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1409/2802] 10.222.0.57 () {58 vars in 1232 bytes} [Fri Jul 1 16:52:39 2022] DELETE /identity/v3/projects/9cc592f9f66a4ffc84a4550586e9a0f4 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 16:52:39.665877 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bedadefb-57c1-440a-b59f-254d3aa37774 service neutron] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:52:39.675879 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bedadefb-57c1-440a-b59f-254d3aa37774 service neutron] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:52:39.694695 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:52:39.736194 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6f2dea-0b07-4303-9a0d-add9d2443f6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:52:39.737062 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6f2dea-0b07-4303-9a0d-add9d2443f6d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:52:39.737370 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6f2dea-0b07-4303-9a0d-add9d2443f6d admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:52:39.737678 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6f2dea-0b07-4303-9a0d-add9d2443f6d admin admin] PATH_INFO: `/v3/users/e2bb569673634c749168d8e443b8a07b` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:52:39.738951 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6f2dea-0b07-4303-9a0d-add9d2443f6d admin admin] RBAC: Authorizing `identity:delete_user(user_id=e2bb569673634c749168d8e443b8a07b)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:52:39.741173 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6f2dea-0b07-4303-9a0d-add9d2443f6d admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:52:39.786561 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-7d6f2dea-0b07-4303-9a0d-add9d2443f6d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e2bb569673634c749168d8e443b8a07b'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:52:39.787131 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-7d6f2dea-0b07-4303-9a0d-add9d2443f6d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e2bb569673634c749168d8e443b8a07b'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:52:39.787767 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-7d6f2dea-0b07-4303-9a0d-add9d2443f6d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e2bb569673634c749168d8e443b8a07b'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:52:39.795095 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-7d6f2dea-0b07-4303-9a0d-add9d2443f6d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e2bb569673634c749168d8e443b8a07b'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:52:39.806902 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-7d6f2dea-0b07-4303-9a0d-add9d2443f6d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e2bb569673634c749168d8e443b8a07b'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:52:39.819434 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1394/2803] 10.222.0.57 () {58 vars in 1223 bytes} [Fri Jul 1 16:52:39 2022] DELETE /identity/v3/users/e2bb569673634c749168d8e443b8a07b => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 16:52:40.103854 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55c6de6-38cc-4ada-b99f-f77efb9cdea7 admin admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:52:40.105309 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-c55c6de6-38cc-4ada-b99f-f77efb9cdea7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:52:40.119269 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55c6de6-38cc-4ada-b99f-f77efb9cdea7 admin admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:52:40.176250 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a821e1-cf52-4057-b680-aba4eb9411bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:52:40.177107 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a821e1-cf52-4057-b680-aba4eb9411bb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:52:40.177388 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a821e1-cf52-4057-b680-aba4eb9411bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:52:40.177664 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a821e1-cf52-4057-b680-aba4eb9411bb admin admin] PATH_INFO: `/v3/projects/81754900e29445e88205dbc82c8d4145` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:52:40.179481 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a821e1-cf52-4057-b680-aba4eb9411bb admin admin] RBAC: Authorizing `identity:delete_project(project_id=81754900e29445e88205dbc82c8d4145)` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:52:40.181381 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a821e1-cf52-4057-b680-aba4eb9411bb admin admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:52:40.214026 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:52:40.232246 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-b4a821e1-cf52-4057-b680-aba4eb9411bb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '81754900e29445e88205dbc82c8d4145', 'request_id': 'req-b4a821e1-cf52-4057-b680-aba4eb9411bb'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:52:40.239483 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-b4a821e1-cf52-4057-b680-aba4eb9411bb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '81754900e29445e88205dbc82c8d4145', 'request_id': 'req-b4a821e1-cf52-4057-b680-aba4eb9411bb'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:52:40.246120 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-b4a821e1-cf52-4057-b680-aba4eb9411bb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '81754900e29445e88205dbc82c8d4145', 'request_id': 'req-b4a821e1-cf52-4057-b680-aba4eb9411bb'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:52:40.257982 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-b4a821e1-cf52-4057-b680-aba4eb9411bb admin admin] The token cache is being invalidate because project 81754900e29445e88205dbc82c8d4145 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80312) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 01 16:52:40.258271 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-b4a821e1-cf52-4057-b680-aba4eb9411bb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:52:40.258955 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-b4a821e1-cf52-4057-b680-aba4eb9411bb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:52:40.260516 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1410/2804] 10.222.0.57 () {58 vars in 1232 bytes} [Fri Jul 1 16:52:40 2022] DELETE /identity/v3/projects/81754900e29445e88205dbc82c8d4145 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 16:52:40.265961 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced1c24e-7b71-4824-a47f-0b1addbd61aa None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:52:40.266496 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced1c24e-7b71-4824-a47f-0b1addbd61aa None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:52:40.266709 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced1c24e-7b71-4824-a47f-0b1addbd61aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:52:40.285902 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-ced1c24e-7b71-4824-a47f-0b1addbd61aa None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:52:40.287125 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-ced1c24e-7b71-4824-a47f-0b1addbd61aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:52:40.346952 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1395/2805] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 1 16:52:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 16:52:40.355619 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d556359-892b-42c4-a15d-1c1d1da05b79 None None] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:52:40.355939 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d556359-892b-42c4-a15d-1c1d1da05b79 None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:52:40.356219 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d556359-892b-42c4-a15d-1c1d1da05b79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:52:40.375206 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.auth.core [None req-9d556359-892b-42c4-a15d-1c1d1da05b79 None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80312) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:52:40.376439 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-9d556359-892b-42c4-a15d-1c1d1da05b79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:52:40.384593 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1411/2806] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 1 16:52:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 16:52:40.388882 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced1c24e-7b71-4824-a47f-0b1addbd61aa None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:52:40.398683 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced1c24e-7b71-4824-a47f-0b1addbd61aa None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:52:40.403582 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8254150-a0fb-49c8-a634-d9fb9dcca67b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:52:40.404259 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8254150-a0fb-49c8-a634-d9fb9dcca67b admin admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:52:40.404540 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8254150-a0fb-49c8-a634-d9fb9dcca67b admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:52:40.404816 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8254150-a0fb-49c8-a634-d9fb9dcca67b admin admin] PATH_INFO: `/v3/domains` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:52:40.405536 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8254150-a0fb-49c8-a634-d9fb9dcca67b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:52:40.405839 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8254150-a0fb-49c8-a634-d9fb9dcca67b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:52:40.407473 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8254150-a0fb-49c8-a634-d9fb9dcca67b admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:52:40.419696 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1396/2807] 10.222.0.57 () {58 vars in 1152 bytes} [Fri Jul 1 16:52:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:52:40.424228 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d556359-892b-42c4-a15d-1c1d1da05b79 None None] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:52:40.435599 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d556359-892b-42c4-a15d-1c1d1da05b79 None None] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:52:40.440270 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb574f1f-2e4e-429c-b0de-79a531cb85f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:52:40.440920 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb574f1f-2e4e-429c-b0de-79a531cb85f9 admin admin] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:52:40.441206 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb574f1f-2e4e-429c-b0de-79a531cb85f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:52:40.441501 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb574f1f-2e4e-429c-b0de-79a531cb85f9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:52:40.442177 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb574f1f-2e4e-429c-b0de-79a531cb85f9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:52:40.444699 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb574f1f-2e4e-429c-b0de-79a531cb85f9 admin admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:52:40.469851 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1412/2808] 10.222.0.57 () {60 vars in 1152 bytes} [Fri Jul 1 16:52:40 2022] POST /identity/v3/projects => generated 387 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 16:52:40.474107 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8254150-a0fb-49c8-a634-d9fb9dcca67b admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:52:40.484555 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8254150-a0fb-49c8-a634-d9fb9dcca67b admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:52:40.510889 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:52:40.551100 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622d29e8-7acc-47c9-b58b-7ca6f5db4df5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:52:40.551945 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622d29e8-7acc-47c9-b58b-7ca6f5db4df5 admin admin] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:52:40.552267 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622d29e8-7acc-47c9-b58b-7ca6f5db4df5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:52:40.552554 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622d29e8-7acc-47c9-b58b-7ca6f5db4df5 admin admin] PATH_INFO: `/v3/users` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:52:40.553259 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-622d29e8-7acc-47c9-b58b-7ca6f5db4df5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:52:40.556016 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-622d29e8-7acc-47c9-b58b-7ca6f5db4df5 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:52:40.577467 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1397/2809] 10.222.0.57 () {60 vars in 1143 bytes} [Fri Jul 1 16:52:40 2022] POST /identity/v3/users => generated 294 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 16:52:40.581795 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb574f1f-2e4e-429c-b0de-79a531cb85f9 admin admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:52:40.592725 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb574f1f-2e4e-429c-b0de-79a531cb85f9 admin admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:52:40.597402 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b025933-f191-4638-a4af-d3cd270e4aad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:52:40.597994 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b025933-f191-4638-a4af-d3cd270e4aad admin admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:52:40.598275 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b025933-f191-4638-a4af-d3cd270e4aad admin admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:52:40.598579 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b025933-f191-4638-a4af-d3cd270e4aad admin admin] PATH_INFO: `/v3/roles` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:52:40.599305 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b025933-f191-4638-a4af-d3cd270e4aad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:52:40.601575 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b025933-f191-4638-a4af-d3cd270e4aad admin admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:52:40.611800 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1413/2810] 10.222.0.57 () {58 vars in 1121 bytes} [Fri Jul 1 16:52:40 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 16:52:40.615717 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622d29e8-7acc-47c9-b58b-7ca6f5db4df5 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:52:40.626262 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622d29e8-7acc-47c9-b58b-7ca6f5db4df5 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:52:40.630903 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2362642-9c8b-49b0-8a77-37d2a2812164 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:52:40.631699 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2362642-9c8b-49b0-8a77-37d2a2812164 admin admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:52:40.631983 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2362642-9c8b-49b0-8a77-37d2a2812164 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:52:40.632259 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2362642-9c8b-49b0-8a77-37d2a2812164 admin admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:52:40.632975 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2362642-9c8b-49b0-8a77-37d2a2812164 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:52:40.634599 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2362642-9c8b-49b0-8a77-37d2a2812164 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:52:40.644436 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1398/2811] 10.222.0.57 () {58 vars in 1121 bytes} [Fri Jul 1 16:52:40 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 16:52:40.648947 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b025933-f191-4638-a4af-d3cd270e4aad admin admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:52:40.659031 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b025933-f191-4638-a4af-d3cd270e4aad admin admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:52:40.665426 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9bcbaa-7055-4fb0-b7ca-628ed113ff90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:52:40.666371 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9bcbaa-7055-4fb0-b7ca-628ed113ff90 admin admin] REQUEST_METHOD: `PUT` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:52:40.666672 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9bcbaa-7055-4fb0-b7ca-628ed113ff90 admin admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:52:40.666955 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9bcbaa-7055-4fb0-b7ca-628ed113ff90 admin admin] PATH_INFO: `/v3/projects/a161a8715c864b80acb8d874c7f8beba/users/2be92527bb0e4a52a18b5704f4df216d/roles/983be40bdab04c478eaa90c6b6a46b14` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:52:40.683024 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d9bcbaa-7055-4fb0-b7ca-628ed113ff90 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a161a8715c864b80acb8d874c7f8beba, user_id=2be92527bb0e4a52a18b5704f4df216d, role_id=983be40bdab04c478eaa90c6b6a46b14)` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:52:40.685756 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d9bcbaa-7055-4fb0-b7ca-628ed113ff90 admin admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:52:40.699200 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1414/2812] 10.222.0.57 () {60 vars in 1482 bytes} [Fri Jul 1 16:52:40 2022] PUT /identity/v3/projects/a161a8715c864b80acb8d874c7f8beba/users/2be92527bb0e4a52a18b5704f4df216d/roles/983be40bdab04c478eaa90c6b6a46b14 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 01 16:52:40.710927 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e49feb7-b0c5-435d-844f-ad334e6f2ca7 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:52:40.711215 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e49feb7-b0c5-435d-844f-ad334e6f2ca7 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:52:40.711520 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e49feb7-b0c5-435d-844f-ad334e6f2ca7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:52:40.738804 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-9e49feb7-b0c5-435d-844f-ad334e6f2ca7 None None] MFA Rules not processed for user `2be92527bb0e4a52a18b5704f4df216d`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:52:40.740022 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-9e49feb7-b0c5-435d-844f-ad334e6f2ca7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:52:40.804044 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1399/2813] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 1 16:52:40 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 16:52:40.808493 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9bcbaa-7055-4fb0-b7ca-628ed113ff90 admin admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:52:40.876235 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9bcbaa-7055-4fb0-b7ca-628ed113ff90 admin admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:52:40.881162 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12c4dcc-9a36-47fc-96af-69f7a9fc68fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:52:40.881997 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12c4dcc-9a36-47fc-96af-69f7a9fc68fe admin admin] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:52:40.882330 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12c4dcc-9a36-47fc-96af-69f7a9fc68fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:52:40.882676 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12c4dcc-9a36-47fc-96af-69f7a9fc68fe admin admin] PATH_INFO: `/v3/projects` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:52:40.883407 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e12c4dcc-9a36-47fc-96af-69f7a9fc68fe admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:52:40.885405 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e12c4dcc-9a36-47fc-96af-69f7a9fc68fe admin admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:52:40.911679 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1415/2814] 10.222.0.57 () {60 vars in 1152 bytes} [Fri Jul 1 16:52:40 2022] POST /identity/v3/projects => generated 387 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 16:52:40.916860 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e49feb7-b0c5-435d-844f-ad334e6f2ca7 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:52:40.927641 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e49feb7-b0c5-435d-844f-ad334e6f2ca7 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:52:40.985446 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5344a08a-8a89-4c23-8866-8fb15e7ce097 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:52:40.986118 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5344a08a-8a89-4c23-8866-8fb15e7ce097 admin admin] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:52:40.986441 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5344a08a-8a89-4c23-8866-8fb15e7ce097 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:52:40.986725 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5344a08a-8a89-4c23-8866-8fb15e7ce097 admin admin] PATH_INFO: `/v3/users` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:52:40.987415 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5344a08a-8a89-4c23-8866-8fb15e7ce097 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:52:40.989337 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5344a08a-8a89-4c23-8866-8fb15e7ce097 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:52:41.010097 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1400/2815] 10.222.0.57 () {60 vars in 1143 bytes} [Fri Jul 1 16:52:40 2022] POST /identity/v3/users => generated 300 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 16:52:41.014137 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12c4dcc-9a36-47fc-96af-69f7a9fc68fe admin admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:52:41.025199 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12c4dcc-9a36-47fc-96af-69f7a9fc68fe admin admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:52:41.029870 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10bf1f95-22bf-48ab-9872-3a6e439f6e4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:52:41.030565 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bf1f95-22bf-48ab-9872-3a6e439f6e4a admin admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:52:41.030852 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bf1f95-22bf-48ab-9872-3a6e439f6e4a admin admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:52:41.031134 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bf1f95-22bf-48ab-9872-3a6e439f6e4a admin admin] PATH_INFO: `/v3/roles` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:52:41.031889 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10bf1f95-22bf-48ab-9872-3a6e439f6e4a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:52:41.033475 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10bf1f95-22bf-48ab-9872-3a6e439f6e4a admin admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:52:41.044915 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1416/2816] 10.222.0.57 () {58 vars in 1121 bytes} [Fri Jul 1 16:52:41 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 16:52:41.049041 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5344a08a-8a89-4c23-8866-8fb15e7ce097 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:52:41.059824 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5344a08a-8a89-4c23-8866-8fb15e7ce097 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:52:41.064483 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd7c328-e0b2-4d02-95e8-577d34e24ff7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:52:41.065390 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd7c328-e0b2-4d02-95e8-577d34e24ff7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:52:41.065669 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd7c328-e0b2-4d02-95e8-577d34e24ff7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:52:41.065943 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd7c328-e0b2-4d02-95e8-577d34e24ff7 admin admin] PATH_INFO: `/v3/projects/997ea01ed07e4541bd72290df7de0abc/users/7f4d7192c68642758cba8ba7e9a3e245/roles/84ee739146934c1ab32c501ff8093644` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:52:41.080684 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd7c328-e0b2-4d02-95e8-577d34e24ff7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=997ea01ed07e4541bd72290df7de0abc, user_id=7f4d7192c68642758cba8ba7e9a3e245, role_id=84ee739146934c1ab32c501ff8093644)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:52:41.083640 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd7c328-e0b2-4d02-95e8-577d34e24ff7 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:52:41.097173 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1401/2817] 10.222.0.57 () {60 vars in 1482 bytes} [Fri Jul 1 16:52:41 2022] PUT /identity/v3/projects/997ea01ed07e4541bd72290df7de0abc/users/7f4d7192c68642758cba8ba7e9a3e245/roles/84ee739146934c1ab32c501ff8093644 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 16:52:41.101282 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10bf1f95-22bf-48ab-9872-3a6e439f6e4a admin admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:52:41.111389 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10bf1f95-22bf-48ab-9872-3a6e439f6e4a admin admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:52:41.132098 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:52:41.171719 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95ed478-bcdb-4e8b-94a0-471b5d20337b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:52:41.172407 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95ed478-bcdb-4e8b-94a0-471b5d20337b admin admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:52:41.172675 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95ed478-bcdb-4e8b-94a0-471b5d20337b admin admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:52:41.172955 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95ed478-bcdb-4e8b-94a0-471b5d20337b admin admin] PATH_INFO: `/v3/roles` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:52:41.173679 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d95ed478-bcdb-4e8b-94a0-471b5d20337b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:52:41.175305 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d95ed478-bcdb-4e8b-94a0-471b5d20337b admin admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:52:41.185192 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1417/2818] 10.222.0.57 () {58 vars in 1121 bytes} [Fri Jul 1 16:52:41 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 16:52:41.189450 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd7c328-e0b2-4d02-95e8-577d34e24ff7 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:52:41.195741 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:52:41.202217 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd7c328-e0b2-4d02-95e8-577d34e24ff7 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:52:41.207691 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815392ba-e065-4660-8fcc-0537368e7c77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:52:41.208599 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815392ba-e065-4660-8fcc-0537368e7c77 admin admin] REQUEST_METHOD: `PUT` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:52:41.208901 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815392ba-e065-4660-8fcc-0537368e7c77 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:52:41.209162 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815392ba-e065-4660-8fcc-0537368e7c77 admin admin] PATH_INFO: `/v3/projects/997ea01ed07e4541bd72290df7de0abc/users/7f4d7192c68642758cba8ba7e9a3e245/roles/84ee739146934c1ab32c501ff8093644` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:52:41.212941 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-815392ba-e065-4660-8fcc-0537368e7c77 admin admin] RBAC: Authorizing `identity:create_grant(project_id=997ea01ed07e4541bd72290df7de0abc, user_id=7f4d7192c68642758cba8ba7e9a3e245, role_id=84ee739146934c1ab32c501ff8093644)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:52:41.216589 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-815392ba-e065-4660-8fcc-0537368e7c77 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:52:41.228263 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1402/2819] 10.222.0.57 () {60 vars in 1482 bytes} [Fri Jul 1 16:52:41 2022] PUT /identity/v3/projects/997ea01ed07e4541bd72290df7de0abc/users/7f4d7192c68642758cba8ba7e9a3e245/roles/84ee739146934c1ab32c501ff8093644 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 16:52:41.238574 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4eb49a-26c0-4bb8-9fca-07cdc56b655f None None] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:52:41.238865 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4eb49a-26c0-4bb8-9fca-07cdc56b655f None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:52:41.239146 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4eb49a-26c0-4bb8-9fca-07cdc56b655f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:52:41.266147 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.auth.core [None req-8a4eb49a-26c0-4bb8-9fca-07cdc56b655f None None] MFA Rules not processed for user `7f4d7192c68642758cba8ba7e9a3e245`. Rule list: `[]` (Enabled: `True`). {{(pid=80312) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:52:41.267375 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-8a4eb49a-26c0-4bb8-9fca-07cdc56b655f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:52:41.333712 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1418/2820] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 1 16:52:41 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 16:52:41.345649 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815392ba-e065-4660-8fcc-0537368e7c77 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:52:41.347340 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-815392ba-e065-4660-8fcc-0537368e7c77 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:52:41.405994 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815392ba-e065-4660-8fcc-0537368e7c77 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:52:41.409534 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6bb93c-6f43-438d-b9c8-f69f901cbcbc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbddf971a2354bbdbb9da1d63f3abf88', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0788e6725a84e57a4174284c1845755', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:52:41.410203 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6bb93c-6f43-438d-b9c8-f69f901cbcbc service cinder] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:52:41.410525 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6bb93c-6f43-438d-b9c8-f69f901cbcbc service cinder] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:52:41.410808 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6bb93c-6f43-438d-b9c8-f69f901cbcbc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:52:41.419956 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c6bb93c-6f43-438d-b9c8-f69f901cbcbc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:52:41.421460 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c6bb93c-6f43-438d-b9c8-f69f901cbcbc service cinder] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:52:41.431241 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1403/2821] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 1 16:52:41 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 16:52:41.504540 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4eb49a-26c0-4bb8-9fca-07cdc56b655f None None] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:52:41.570907 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4eb49a-26c0-4bb8-9fca-07cdc56b655f None None] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:52:41.575695 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8dc302-d449-445c-a48d-b8a0ad2ce14c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:52:41.576624 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8dc302-d449-445c-a48d-b8a0ad2ce14c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:52:41.576936 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8dc302-d449-445c-a48d-b8a0ad2ce14c admin admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:52:41.577243 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8dc302-d449-445c-a48d-b8a0ad2ce14c admin admin] PATH_INFO: `/v3/users/2be92527bb0e4a52a18b5704f4df216d` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:52:41.578494 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd8dc302-d449-445c-a48d-b8a0ad2ce14c admin admin] RBAC: Authorizing `identity:delete_user(user_id=2be92527bb0e4a52a18b5704f4df216d)` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:52:41.580458 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd8dc302-d449-445c-a48d-b8a0ad2ce14c admin admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:52:41.627255 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-fd8dc302-d449-445c-a48d-b8a0ad2ce14c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2be92527bb0e4a52a18b5704f4df216d'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:52:41.628388 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:52:41.641745 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-fd8dc302-d449-445c-a48d-b8a0ad2ce14c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2be92527bb0e4a52a18b5704f4df216d'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:52:41.649441 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-fd8dc302-d449-445c-a48d-b8a0ad2ce14c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2be92527bb0e4a52a18b5704f4df216d'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:52:41.650163 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-fd8dc302-d449-445c-a48d-b8a0ad2ce14c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2be92527bb0e4a52a18b5704f4df216d'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:52:41.650821 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-fd8dc302-d449-445c-a48d-b8a0ad2ce14c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2be92527bb0e4a52a18b5704f4df216d'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:52:41.665244 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1419/2822] 10.222.0.57 () {58 vars in 1223 bytes} [Fri Jul 1 16:52:41 2022] DELETE /identity/v3/users/2be92527bb0e4a52a18b5704f4df216d => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 16:52:41.669896 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7710d9-e903-4bd1-935f-ad337ff5c301 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:52:41.670236 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7710d9-e903-4bd1-935f-ad337ff5c301 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:52:41.670551 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7710d9-e903-4bd1-935f-ad337ff5c301 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:52:41.687995 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-4c7710d9-e903-4bd1-935f-ad337ff5c301 None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:52:41.689166 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-4c7710d9-e903-4bd1-935f-ad337ff5c301 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:52:41.748768 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1404/2823] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 1 16:52:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 16:52:41.758917 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8dc302-d449-445c-a48d-b8a0ad2ce14c admin admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:52:41.760357 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-fd8dc302-d449-445c-a48d-b8a0ad2ce14c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:52:41.818052 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8dc302-d449-445c-a48d-b8a0ad2ce14c admin admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:52:41.821916 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19998c1-c959-4a2a-b75d-ca9f9d630bd7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ed200ce29684c5f80354fa9a8638fe2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0788e6725a84e57a4174284c1845755', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:52:41.822598 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19998c1-c959-4a2a-b75d-ca9f9d630bd7 service neutron] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:52:41.822891 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19998c1-c959-4a2a-b75d-ca9f9d630bd7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:52:41.823170 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19998c1-c959-4a2a-b75d-ca9f9d630bd7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:52:41.833140 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a19998c1-c959-4a2a-b75d-ca9f9d630bd7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:52:41.835179 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a19998c1-c959-4a2a-b75d-ca9f9d630bd7 service neutron] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:52:41.845639 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1420/2824] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 1 16:52:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 01 16:52:42.117941 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7710d9-e903-4bd1-935f-ad337ff5c301 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:52:42.119440 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-4c7710d9-e903-4bd1-935f-ad337ff5c301 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:52:42.131308 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7710d9-e903-4bd1-935f-ad337ff5c301 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:52:42.137782 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce31d663-4029-4815-8570-00dbd276a762 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:52:42.138804 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce31d663-4029-4815-8570-00dbd276a762 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:52:42.139108 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce31d663-4029-4815-8570-00dbd276a762 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:52:42.139400 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce31d663-4029-4815-8570-00dbd276a762 admin admin] PATH_INFO: `/v3/projects/a161a8715c864b80acb8d874c7f8beba` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:52:42.140662 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce31d663-4029-4815-8570-00dbd276a762 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a161a8715c864b80acb8d874c7f8beba)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:52:42.142601 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce31d663-4029-4815-8570-00dbd276a762 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:52:42.191082 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-ce31d663-4029-4815-8570-00dbd276a762 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a161a8715c864b80acb8d874c7f8beba', 'request_id': 'req-ce31d663-4029-4815-8570-00dbd276a762'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:52:42.197509 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-ce31d663-4029-4815-8570-00dbd276a762 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a161a8715c864b80acb8d874c7f8beba', 'request_id': 'req-ce31d663-4029-4815-8570-00dbd276a762'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:52:42.203581 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-ce31d663-4029-4815-8570-00dbd276a762 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a161a8715c864b80acb8d874c7f8beba', 'request_id': 'req-ce31d663-4029-4815-8570-00dbd276a762'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:52:42.216444 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-ce31d663-4029-4815-8570-00dbd276a762 admin admin] The token cache is being invalidate because project a161a8715c864b80acb8d874c7f8beba was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80311) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 01 16:52:42.216750 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-ce31d663-4029-4815-8570-00dbd276a762 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:52:42.217386 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-ce31d663-4029-4815-8570-00dbd276a762 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:52:42.218968 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1405/2825] 10.222.0.57 () {58 vars in 1232 bytes} [Fri Jul 1 16:52:42 2022] DELETE /identity/v3/projects/a161a8715c864b80acb8d874c7f8beba => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 16:52:42.222934 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19998c1-c959-4a2a-b75d-ca9f9d630bd7 service neutron] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:52:42.234079 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19998c1-c959-4a2a-b75d-ca9f9d630bd7 service neutron] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:52:42.299681 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792f4bdf-0cda-4537-b2f6-48079a5b12b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:52:42.300557 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792f4bdf-0cda-4537-b2f6-48079a5b12b8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:52:42.300844 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792f4bdf-0cda-4537-b2f6-48079a5b12b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:52:42.301126 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792f4bdf-0cda-4537-b2f6-48079a5b12b8 admin admin] PATH_INFO: `/v3/users/7f4d7192c68642758cba8ba7e9a3e245` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:52:42.302305 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-792f4bdf-0cda-4537-b2f6-48079a5b12b8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7f4d7192c68642758cba8ba7e9a3e245)` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:52:42.304862 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-792f4bdf-0cda-4537-b2f6-48079a5b12b8 admin admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:52:42.350636 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-792f4bdf-0cda-4537-b2f6-48079a5b12b8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7f4d7192c68642758cba8ba7e9a3e245'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:52:42.362744 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-792f4bdf-0cda-4537-b2f6-48079a5b12b8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7f4d7192c68642758cba8ba7e9a3e245'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:52:42.367826 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:52:42.371568 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-792f4bdf-0cda-4537-b2f6-48079a5b12b8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7f4d7192c68642758cba8ba7e9a3e245'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:52:42.372464 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-792f4bdf-0cda-4537-b2f6-48079a5b12b8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7f4d7192c68642758cba8ba7e9a3e245'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:52:42.373334 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-792f4bdf-0cda-4537-b2f6-48079a5b12b8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7f4d7192c68642758cba8ba7e9a3e245'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:52:42.388757 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1421/2826] 10.222.0.57 () {58 vars in 1223 bytes} [Fri Jul 1 16:52:42 2022] DELETE /identity/v3/users/7f4d7192c68642758cba8ba7e9a3e245 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 16:52:42.679387 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce31d663-4029-4815-8570-00dbd276a762 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:52:42.680859 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-ce31d663-4029-4815-8570-00dbd276a762 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:52:42.693697 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce31d663-4029-4815-8570-00dbd276a762 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:52:42.706944 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:52:42.756940 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ce8424-e1c9-40cc-b793-64bbe73b8e26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:52:42.757786 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ce8424-e1c9-40cc-b793-64bbe73b8e26 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:52:42.758070 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ce8424-e1c9-40cc-b793-64bbe73b8e26 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:52:42.758436 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ce8424-e1c9-40cc-b793-64bbe73b8e26 admin admin] PATH_INFO: `/v3/projects/997ea01ed07e4541bd72290df7de0abc` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:52:42.759765 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ce8424-e1c9-40cc-b793-64bbe73b8e26 admin admin] RBAC: Authorizing `identity:delete_project(project_id=997ea01ed07e4541bd72290df7de0abc)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:52:42.761797 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ce8424-e1c9-40cc-b793-64bbe73b8e26 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:52:42.807568 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-56ce8424-e1c9-40cc-b793-64bbe73b8e26 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '997ea01ed07e4541bd72290df7de0abc', 'request_id': 'req-56ce8424-e1c9-40cc-b793-64bbe73b8e26'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:52:42.812952 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-56ce8424-e1c9-40cc-b793-64bbe73b8e26 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '997ea01ed07e4541bd72290df7de0abc', 'request_id': 'req-56ce8424-e1c9-40cc-b793-64bbe73b8e26'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:52:42.819509 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-56ce8424-e1c9-40cc-b793-64bbe73b8e26 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '997ea01ed07e4541bd72290df7de0abc', 'request_id': 'req-56ce8424-e1c9-40cc-b793-64bbe73b8e26'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:52:42.832019 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-56ce8424-e1c9-40cc-b793-64bbe73b8e26 admin admin] The token cache is being invalidate because project 997ea01ed07e4541bd72290df7de0abc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80311) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 01 16:52:42.832310 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-56ce8424-e1c9-40cc-b793-64bbe73b8e26 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:52:42.833210 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-56ce8424-e1c9-40cc-b793-64bbe73b8e26 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:52:42.834911 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1406/2827] 10.222.0.57 () {58 vars in 1232 bytes} [Fri Jul 1 16:52:42 2022] DELETE /identity/v3/projects/997ea01ed07e4541bd72290df7de0abc => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 16:52:42.840515 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f47f4ba-20c5-447f-bbb9-884ab6b7a52a None None] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:52:42.840805 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f47f4ba-20c5-447f-bbb9-884ab6b7a52a None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:52:42.841094 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f47f4ba-20c5-447f-bbb9-884ab6b7a52a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:52:42.861603 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.auth.core [None req-6f47f4ba-20c5-447f-bbb9-884ab6b7a52a None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80312) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:52:42.862858 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-6f47f4ba-20c5-447f-bbb9-884ab6b7a52a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:52:42.935674 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1422/2828] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 1 16:52:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 01 16:52:42.943564 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8a6f6c-5348-4ca0-bdd7-a558c2afe34c None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:52:42.944393 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8a6f6c-5348-4ca0-bdd7-a558c2afe34c None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:52:42.944739 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8a6f6c-5348-4ca0-bdd7-a558c2afe34c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:52:42.964076 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-5f8a6f6c-5348-4ca0-bdd7-a558c2afe34c None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:52:42.965281 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-5f8a6f6c-5348-4ca0-bdd7-a558c2afe34c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:52:42.973236 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1407/2829] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 1 16:52:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 01 16:52:42.978057 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f47f4ba-20c5-447f-bbb9-884ab6b7a52a None None] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:52:42.987985 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f47f4ba-20c5-447f-bbb9-884ab6b7a52a None None] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:52:42.993833 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad1cae0-3b9b-47f7-a4c0-868c20fdc869 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:52:42.994525 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad1cae0-3b9b-47f7-a4c0-868c20fdc869 admin admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:52:42.994819 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad1cae0-3b9b-47f7-a4c0-868c20fdc869 admin admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:52:42.995097 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad1cae0-3b9b-47f7-a4c0-868c20fdc869 admin admin] PATH_INFO: `/v3/domains` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:52:42.995828 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad1cae0-3b9b-47f7-a4c0-868c20fdc869 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80312) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:52:42.996140 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad1cae0-3b9b-47f7-a4c0-868c20fdc869 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:52:42.997758 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad1cae0-3b9b-47f7-a4c0-868c20fdc869 admin admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:52:43.010473 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1423/2830] 10.222.0.57 () {58 vars in 1152 bytes} [Fri Jul 1 16:52:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:52:43.015970 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8a6f6c-5348-4ca0-bdd7-a558c2afe34c None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:52:43.025681 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8a6f6c-5348-4ca0-bdd7-a558c2afe34c None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:52:43.030262 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74d35f8-6f42-4484-a0cd-e07da0071de1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:52:43.030940 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74d35f8-6f42-4484-a0cd-e07da0071de1 admin admin] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:52:43.031222 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74d35f8-6f42-4484-a0cd-e07da0071de1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:52:43.031497 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74d35f8-6f42-4484-a0cd-e07da0071de1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:52:43.032189 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e74d35f8-6f42-4484-a0cd-e07da0071de1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:52:43.034108 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e74d35f8-6f42-4484-a0cd-e07da0071de1 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:52:43.062096 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1408/2831] 10.222.0.57 () {60 vars in 1152 bytes} [Fri Jul 1 16:52:43 2022] POST /identity/v3/projects => generated 381 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 16:52:43.064324 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad1cae0-3b9b-47f7-a4c0-868c20fdc869 admin admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:52:43.075152 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad1cae0-3b9b-47f7-a4c0-868c20fdc869 admin admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:52:43.132632 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1c1ada-3327-4b29-a6d7-b46f8ea6cbe6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:52:43.133296 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1c1ada-3327-4b29-a6d7-b46f8ea6cbe6 admin admin] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:52:43.133579 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1c1ada-3327-4b29-a6d7-b46f8ea6cbe6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:52:43.133858 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1c1ada-3327-4b29-a6d7-b46f8ea6cbe6 admin admin] PATH_INFO: `/v3/users` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:52:43.134570 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b1c1ada-3327-4b29-a6d7-b46f8ea6cbe6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:52:43.136492 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b1c1ada-3327-4b29-a6d7-b46f8ea6cbe6 admin admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:52:43.156250 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1424/2832] 10.222.0.57 () {60 vars in 1143 bytes} [Fri Jul 1 16:52:43 2022] POST /identity/v3/users => generated 291 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 16:52:43.160549 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74d35f8-6f42-4484-a0cd-e07da0071de1 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:52:43.172048 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74d35f8-6f42-4484-a0cd-e07da0071de1 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:52:43.176722 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997e372a-9821-4c8e-93be-789bfaff70f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:52:43.177408 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997e372a-9821-4c8e-93be-789bfaff70f6 admin admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:52:43.177697 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997e372a-9821-4c8e-93be-789bfaff70f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:52:43.178022 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997e372a-9821-4c8e-93be-789bfaff70f6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:52:43.179508 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997e372a-9821-4c8e-93be-789bfaff70f6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:52:43.181158 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997e372a-9821-4c8e-93be-789bfaff70f6 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:52:43.191598 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1409/2833] 10.222.0.57 () {58 vars in 1121 bytes} [Fri Jul 1 16:52:43 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 16:52:43.196224 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1c1ada-3327-4b29-a6d7-b46f8ea6cbe6 admin admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:52:43.205931 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1c1ada-3327-4b29-a6d7-b46f8ea6cbe6 admin admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:52:43.211081 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10dee7a-89cf-4bef-8b6e-f3e822b2d779 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:52:43.211728 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10dee7a-89cf-4bef-8b6e-f3e822b2d779 admin admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:52:43.212010 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10dee7a-89cf-4bef-8b6e-f3e822b2d779 admin admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:52:43.212285 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10dee7a-89cf-4bef-8b6e-f3e822b2d779 admin admin] PATH_INFO: `/v3/roles` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:52:43.213008 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d10dee7a-89cf-4bef-8b6e-f3e822b2d779 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:52:43.214612 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d10dee7a-89cf-4bef-8b6e-f3e822b2d779 admin admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:52:43.224431 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1425/2834] 10.222.0.57 () {58 vars in 1121 bytes} [Fri Jul 1 16:52:43 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 01 16:52:43.228495 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997e372a-9821-4c8e-93be-789bfaff70f6 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:52:43.238567 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997e372a-9821-4c8e-93be-789bfaff70f6 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:52:43.244830 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe56071-539c-4b3c-b3a0-39e1ce898e2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:52:43.245990 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe56071-539c-4b3c-b3a0-39e1ce898e2b admin admin] REQUEST_METHOD: `PUT` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:52:43.246276 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe56071-539c-4b3c-b3a0-39e1ce898e2b admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:52:43.246659 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe56071-539c-4b3c-b3a0-39e1ce898e2b admin admin] PATH_INFO: `/v3/projects/a968ac7d79ab4139a70124f49570c00a/users/b5d9b16029564042b8cafab88b74fec2/roles/983be40bdab04c478eaa90c6b6a46b14` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:52:43.262770 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe56071-539c-4b3c-b3a0-39e1ce898e2b admin admin] RBAC: Authorizing `identity:create_grant(project_id=a968ac7d79ab4139a70124f49570c00a, user_id=b5d9b16029564042b8cafab88b74fec2, role_id=983be40bdab04c478eaa90c6b6a46b14)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:52:43.265335 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe56071-539c-4b3c-b3a0-39e1ce898e2b admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:52:43.278622 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1410/2835] 10.222.0.57 () {60 vars in 1482 bytes} [Fri Jul 1 16:52:43 2022] PUT /identity/v3/projects/a968ac7d79ab4139a70124f49570c00a/users/b5d9b16029564042b8cafab88b74fec2/roles/983be40bdab04c478eaa90c6b6a46b14 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 16:52:43.341332 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d8cdd4-2209-45cf-8a3b-ef3504df2c55 None None] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:52:43.341632 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d8cdd4-2209-45cf-8a3b-ef3504df2c55 None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:52:43.341919 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d8cdd4-2209-45cf-8a3b-ef3504df2c55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:52:43.371075 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.auth.core [None req-50d8cdd4-2209-45cf-8a3b-ef3504df2c55 None None] MFA Rules not processed for user `b5d9b16029564042b8cafab88b74fec2`. Rule list: `[]` (Enabled: `True`). {{(pid=80312) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:52:43.372304 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-50d8cdd4-2209-45cf-8a3b-ef3504df2c55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:52:43.413422 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=95,keystone:INSERT=4,keystone:DELETE=1 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:52:43.438284 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1426/2836] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 1 16:52:43 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 16:52:43.442971 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe56071-539c-4b3c-b3a0-39e1ce898e2b admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:52:43.487721 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:52:43.506833 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe56071-539c-4b3c-b3a0-39e1ce898e2b admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:52:43.511746 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df436ee9-2ee7-416d-bb83-9cd91a22f6f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:52:43.512408 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df436ee9-2ee7-416d-bb83-9cd91a22f6f3 admin admin] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:52:43.512696 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df436ee9-2ee7-416d-bb83-9cd91a22f6f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:52:43.513045 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df436ee9-2ee7-416d-bb83-9cd91a22f6f3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:52:43.513686 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df436ee9-2ee7-416d-bb83-9cd91a22f6f3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:52:43.515612 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df436ee9-2ee7-416d-bb83-9cd91a22f6f3 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:52:43.544223 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1411/2837] 10.222.0.57 () {60 vars in 1152 bytes} [Fri Jul 1 16:52:43 2022] POST /identity/v3/projects => generated 381 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 16:52:43.548298 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d8cdd4-2209-45cf-8a3b-ef3504df2c55 None None] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:52:43.559779 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d8cdd4-2209-45cf-8a3b-ef3504df2c55 None None] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:52:43.614376 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685fb180-9d6e-433a-9c10-23c2782c5da1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:52:43.615046 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685fb180-9d6e-433a-9c10-23c2782c5da1 admin admin] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:52:43.615350 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685fb180-9d6e-433a-9c10-23c2782c5da1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:52:43.615609 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685fb180-9d6e-433a-9c10-23c2782c5da1 admin admin] PATH_INFO: `/v3/users` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:52:43.617209 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-685fb180-9d6e-433a-9c10-23c2782c5da1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:52:43.619088 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-685fb180-9d6e-433a-9c10-23c2782c5da1 admin admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:52:43.637969 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1427/2838] 10.222.0.57 () {60 vars in 1143 bytes} [Fri Jul 1 16:52:43 2022] POST /identity/v3/users => generated 297 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 16:52:43.642107 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df436ee9-2ee7-416d-bb83-9cd91a22f6f3 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:52:43.652356 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df436ee9-2ee7-416d-bb83-9cd91a22f6f3 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:52:43.657003 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22fc6dc3-86bf-498d-9047-b0bdac587d00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:52:43.657654 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fc6dc3-86bf-498d-9047-b0bdac587d00 admin admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:52:43.657937 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fc6dc3-86bf-498d-9047-b0bdac587d00 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:52:43.658216 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fc6dc3-86bf-498d-9047-b0bdac587d00 admin admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:52:43.658976 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22fc6dc3-86bf-498d-9047-b0bdac587d00 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:52:43.660642 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22fc6dc3-86bf-498d-9047-b0bdac587d00 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:52:43.673128 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1412/2839] 10.222.0.57 () {58 vars in 1121 bytes} [Fri Jul 1 16:52:43 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 16:52:43.677260 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685fb180-9d6e-433a-9c10-23c2782c5da1 admin admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:52:43.687741 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685fb180-9d6e-433a-9c10-23c2782c5da1 admin admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:52:43.692250 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518eee48-0270-4885-b4c5-f89de8dc3e2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:52:43.693119 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518eee48-0270-4885-b4c5-f89de8dc3e2e admin admin] REQUEST_METHOD: `PUT` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:52:43.693399 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518eee48-0270-4885-b4c5-f89de8dc3e2e admin admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:52:43.693674 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518eee48-0270-4885-b4c5-f89de8dc3e2e admin admin] PATH_INFO: `/v3/projects/4d28cd516c3c4819be0fcb1ed2eb3bb3/users/54be2cc82ff544e6a47579c2d98008b4/roles/84ee739146934c1ab32c501ff8093644` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:52:43.708904 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-518eee48-0270-4885-b4c5-f89de8dc3e2e admin admin] RBAC: Authorizing `identity:create_grant(project_id=4d28cd516c3c4819be0fcb1ed2eb3bb3, user_id=54be2cc82ff544e6a47579c2d98008b4, role_id=84ee739146934c1ab32c501ff8093644)` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:52:43.712172 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-518eee48-0270-4885-b4c5-f89de8dc3e2e admin admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:52:43.725290 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1428/2840] 10.222.0.57 () {60 vars in 1482 bytes} [Fri Jul 1 16:52:43 2022] PUT /identity/v3/projects/4d28cd516c3c4819be0fcb1ed2eb3bb3/users/54be2cc82ff544e6a47579c2d98008b4/roles/84ee739146934c1ab32c501ff8093644 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 16:52:43.729624 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22fc6dc3-86bf-498d-9047-b0bdac587d00 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:52:43.739631 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22fc6dc3-86bf-498d-9047-b0bdac587d00 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:52:43.795643 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72c31b2-395b-4035-b278-66fb90babab4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:52:43.796310 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72c31b2-395b-4035-b278-66fb90babab4 admin admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:52:43.796591 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72c31b2-395b-4035-b278-66fb90babab4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:52:43.796868 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72c31b2-395b-4035-b278-66fb90babab4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:52:43.797589 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f72c31b2-395b-4035-b278-66fb90babab4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:52:43.799207 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f72c31b2-395b-4035-b278-66fb90babab4 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:52:43.809175 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1413/2841] 10.222.0.57 () {58 vars in 1121 bytes} [Fri Jul 1 16:52:43 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 16:52:43.813542 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518eee48-0270-4885-b4c5-f89de8dc3e2e admin admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:52:43.823807 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518eee48-0270-4885-b4c5-f89de8dc3e2e admin admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:52:43.828401 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe368c44-dd84-428c-a4e4-86d8b6bd869e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:52:43.829271 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe368c44-dd84-428c-a4e4-86d8b6bd869e admin admin] REQUEST_METHOD: `PUT` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:52:43.829549 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe368c44-dd84-428c-a4e4-86d8b6bd869e admin admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:52:43.829824 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe368c44-dd84-428c-a4e4-86d8b6bd869e admin admin] PATH_INFO: `/v3/projects/4d28cd516c3c4819be0fcb1ed2eb3bb3/users/54be2cc82ff544e6a47579c2d98008b4/roles/84ee739146934c1ab32c501ff8093644` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:52:43.831828 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe368c44-dd84-428c-a4e4-86d8b6bd869e admin admin] RBAC: Authorizing `identity:create_grant(project_id=4d28cd516c3c4819be0fcb1ed2eb3bb3, user_id=54be2cc82ff544e6a47579c2d98008b4, role_id=84ee739146934c1ab32c501ff8093644)` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:52:43.835315 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe368c44-dd84-428c-a4e4-86d8b6bd869e admin admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:52:43.846159 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1429/2842] 10.222.0.57 () {60 vars in 1482 bytes} [Fri Jul 1 16:52:43 2022] PUT /identity/v3/projects/4d28cd516c3c4819be0fcb1ed2eb3bb3/users/54be2cc82ff544e6a47579c2d98008b4/roles/84ee739146934c1ab32c501ff8093644 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 16:52:43.857861 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ddd57b-b8c8-43c6-928d-8ffc905ac9f5 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:52:43.858180 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ddd57b-b8c8-43c6-928d-8ffc905ac9f5 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:52:43.858591 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ddd57b-b8c8-43c6-928d-8ffc905ac9f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:52:43.884765 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-78ddd57b-b8c8-43c6-928d-8ffc905ac9f5 None None] MFA Rules not processed for user `54be2cc82ff544e6a47579c2d98008b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:52:43.885985 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-78ddd57b-b8c8-43c6-928d-8ffc905ac9f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:52:43.954076 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1414/2843] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 1 16:52:43 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 16:52:43.966239 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe368c44-dd84-428c-a4e4-86d8b6bd869e admin admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:52:43.967944 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-fe368c44-dd84-428c-a4e4-86d8b6bd869e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:52:44.024429 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe368c44-dd84-428c-a4e4-86d8b6bd869e admin admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:52:44.028128 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7f38a8-ea80-4fad-abeb-5f5a3f432423 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbddf971a2354bbdbb9da1d63f3abf88', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0788e6725a84e57a4174284c1845755', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:52:44.028763 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7f38a8-ea80-4fad-abeb-5f5a3f432423 service cinder] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:52:44.029045 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7f38a8-ea80-4fad-abeb-5f5a3f432423 service cinder] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:52:44.029319 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7f38a8-ea80-4fad-abeb-5f5a3f432423 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:52:44.039094 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b7f38a8-ea80-4fad-abeb-5f5a3f432423 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:52:44.040589 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b7f38a8-ea80-4fad-abeb-5f5a3f432423 service cinder] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:52:44.049900 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1430/2844] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 1 16:52:43 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 16:52:44.119949 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ddd57b-b8c8-43c6-928d-8ffc905ac9f5 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:52:44.129580 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ddd57b-b8c8-43c6-928d-8ffc905ac9f5 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:52:44.133848 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b3e17f-ad93-41ac-8996-b9d958ea5b58 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbddf971a2354bbdbb9da1d63f3abf88', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0788e6725a84e57a4174284c1845755', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:52:44.134518 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b3e17f-ad93-41ac-8996-b9d958ea5b58 service cinder] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:52:44.134805 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b3e17f-ad93-41ac-8996-b9d958ea5b58 service cinder] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:52:44.135086 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b3e17f-ad93-41ac-8996-b9d958ea5b58 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:52:44.168668 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:52:44.193113 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b3e17f-ad93-41ac-8996-b9d958ea5b58 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:52:44.195113 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b3e17f-ad93-41ac-8996-b9d958ea5b58 service cinder] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:52:44.204081 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1415/2845] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 1 16:52:44 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 16:52:44.460421 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7f38a8-ea80-4fad-abeb-5f5a3f432423 service cinder] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:52:44.500421 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:52:44.544529 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7f38a8-ea80-4fad-abeb-5f5a3f432423 service cinder] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:52:44.549019 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133bd702-9aed-41f5-8d67-a1435a1f2ef6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:52:44.549885 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133bd702-9aed-41f5-8d67-a1435a1f2ef6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:52:44.550167 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133bd702-9aed-41f5-8d67-a1435a1f2ef6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:52:44.550473 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133bd702-9aed-41f5-8d67-a1435a1f2ef6 admin admin] PATH_INFO: `/v3/users/b5d9b16029564042b8cafab88b74fec2` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:52:44.551652 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-133bd702-9aed-41f5-8d67-a1435a1f2ef6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b5d9b16029564042b8cafab88b74fec2)` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:52:44.554181 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-133bd702-9aed-41f5-8d67-a1435a1f2ef6 admin admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:52:44.601386 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-133bd702-9aed-41f5-8d67-a1435a1f2ef6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b5d9b16029564042b8cafab88b74fec2'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:52:44.613448 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-133bd702-9aed-41f5-8d67-a1435a1f2ef6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b5d9b16029564042b8cafab88b74fec2'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:52:44.620472 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-133bd702-9aed-41f5-8d67-a1435a1f2ef6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b5d9b16029564042b8cafab88b74fec2'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:52:44.621189 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-133bd702-9aed-41f5-8d67-a1435a1f2ef6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b5d9b16029564042b8cafab88b74fec2'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:52:44.621901 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-133bd702-9aed-41f5-8d67-a1435a1f2ef6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b5d9b16029564042b8cafab88b74fec2'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:52:44.635971 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1431/2846] 10.222.0.57 () {58 vars in 1223 bytes} [Fri Jul 1 16:52:44 2022] DELETE /identity/v3/users/b5d9b16029564042b8cafab88b74fec2 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 16:52:44.640584 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffacbbf4-63d9-4493-a1d6-d8c682daafbc None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:52:44.640876 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffacbbf4-63d9-4493-a1d6-d8c682daafbc None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:52:44.641158 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffacbbf4-63d9-4493-a1d6-d8c682daafbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:52:44.660727 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-ffacbbf4-63d9-4493-a1d6-d8c682daafbc None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:52:44.662616 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-ffacbbf4-63d9-4493-a1d6-d8c682daafbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:52:44.727732 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1416/2847] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 1 16:52:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 16:52:44.737368 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133bd702-9aed-41f5-8d67-a1435a1f2ef6 admin admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:52:44.738826 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-133bd702-9aed-41f5-8d67-a1435a1f2ef6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:52:44.800954 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133bd702-9aed-41f5-8d67-a1435a1f2ef6 admin admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:52:44.805831 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638f2477-55fd-4983-8d67-2dbb705ca4f0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ed200ce29684c5f80354fa9a8638fe2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0788e6725a84e57a4174284c1845755', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:52:44.806448 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638f2477-55fd-4983-8d67-2dbb705ca4f0 service neutron] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:52:44.806734 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638f2477-55fd-4983-8d67-2dbb705ca4f0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:52:44.807011 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638f2477-55fd-4983-8d67-2dbb705ca4f0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:52:44.816381 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638f2477-55fd-4983-8d67-2dbb705ca4f0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:52:44.817814 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638f2477-55fd-4983-8d67-2dbb705ca4f0 service neutron] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:52:44.827842 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1432/2848] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 1 16:52:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 16:52:45.100057 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffacbbf4-63d9-4493-a1d6-d8c682daafbc None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:52:45.102198 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-ffacbbf4-63d9-4493-a1d6-d8c682daafbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:52:45.115067 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffacbbf4-63d9-4493-a1d6-d8c682daafbc None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:52:45.120098 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19e92a4-d0fd-4e6f-b10f-bc575e631dcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:52:45.121431 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19e92a4-d0fd-4e6f-b10f-bc575e631dcc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:52:45.121431 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19e92a4-d0fd-4e6f-b10f-bc575e631dcc admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:52:45.121678 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19e92a4-d0fd-4e6f-b10f-bc575e631dcc admin admin] PATH_INFO: `/v3/projects/a968ac7d79ab4139a70124f49570c00a` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:52:45.122842 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19e92a4-d0fd-4e6f-b10f-bc575e631dcc admin admin] RBAC: Authorizing `identity:delete_project(project_id=a968ac7d79ab4139a70124f49570c00a)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:52:45.124781 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19e92a4-d0fd-4e6f-b10f-bc575e631dcc admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:52:45.172149 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-b19e92a4-d0fd-4e6f-b10f-bc575e631dcc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a968ac7d79ab4139a70124f49570c00a', 'request_id': 'req-b19e92a4-d0fd-4e6f-b10f-bc575e631dcc'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:52:45.177388 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-b19e92a4-d0fd-4e6f-b10f-bc575e631dcc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a968ac7d79ab4139a70124f49570c00a', 'request_id': 'req-b19e92a4-d0fd-4e6f-b10f-bc575e631dcc'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:52:45.183687 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-b19e92a4-d0fd-4e6f-b10f-bc575e631dcc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a968ac7d79ab4139a70124f49570c00a', 'request_id': 'req-b19e92a4-d0fd-4e6f-b10f-bc575e631dcc'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:52:45.196515 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-b19e92a4-d0fd-4e6f-b10f-bc575e631dcc admin admin] The token cache is being invalidate because project a968ac7d79ab4139a70124f49570c00a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80311) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 01 16:52:45.196804 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-b19e92a4-d0fd-4e6f-b10f-bc575e631dcc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:52:45.197412 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-b19e92a4-d0fd-4e6f-b10f-bc575e631dcc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:52:45.198978 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1417/2849] 10.222.0.57 () {58 vars in 1232 bytes} [Fri Jul 1 16:52:45 2022] DELETE /identity/v3/projects/a968ac7d79ab4139a70124f49570c00a => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 16:52:45.202930 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638f2477-55fd-4983-8d67-2dbb705ca4f0 service neutron] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:52:45.213555 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638f2477-55fd-4983-8d67-2dbb705ca4f0 service neutron] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:52:45.246477 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:52:45.273206 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b69ae5-ac5f-4d25-af44-748385e1b88d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:52:45.274073 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b69ae5-ac5f-4d25-af44-748385e1b88d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:52:45.274387 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b69ae5-ac5f-4d25-af44-748385e1b88d admin admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:52:45.274655 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b69ae5-ac5f-4d25-af44-748385e1b88d admin admin] PATH_INFO: `/v3/users/54be2cc82ff544e6a47579c2d98008b4` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:52:45.275871 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b69ae5-ac5f-4d25-af44-748385e1b88d admin admin] RBAC: Authorizing `identity:delete_user(user_id=54be2cc82ff544e6a47579c2d98008b4)` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:52:45.277789 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b69ae5-ac5f-4d25-af44-748385e1b88d admin admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:52:45.324548 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-10b69ae5-ac5f-4d25-af44-748385e1b88d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '54be2cc82ff544e6a47579c2d98008b4'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:52:45.337092 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-10b69ae5-ac5f-4d25-af44-748385e1b88d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '54be2cc82ff544e6a47579c2d98008b4'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:52:45.344054 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-10b69ae5-ac5f-4d25-af44-748385e1b88d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '54be2cc82ff544e6a47579c2d98008b4'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:52:45.344690 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-10b69ae5-ac5f-4d25-af44-748385e1b88d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '54be2cc82ff544e6a47579c2d98008b4'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:52:45.345289 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-10b69ae5-ac5f-4d25-af44-748385e1b88d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '54be2cc82ff544e6a47579c2d98008b4'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:52:45.357644 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1433/2850] 10.222.0.57 () {58 vars in 1223 bytes} [Fri Jul 1 16:52:45 2022] DELETE /identity/v3/users/54be2cc82ff544e6a47579c2d98008b4 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 16:52:45.633657 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19e92a4-d0fd-4e6f-b10f-bc575e631dcc admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:52:45.635435 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-b19e92a4-d0fd-4e6f-b10f-bc575e631dcc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:52:45.661776 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19e92a4-d0fd-4e6f-b10f-bc575e631dcc admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:52:45.709362 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:52:45.724113 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a550cd-af1a-4cf9-8870-250baab6dd19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:52:45.724979 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a550cd-af1a-4cf9-8870-250baab6dd19 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:52:45.725485 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a550cd-af1a-4cf9-8870-250baab6dd19 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:52:45.725767 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a550cd-af1a-4cf9-8870-250baab6dd19 admin admin] PATH_INFO: `/v3/projects/4d28cd516c3c4819be0fcb1ed2eb3bb3` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:52:45.726980 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a550cd-af1a-4cf9-8870-250baab6dd19 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4d28cd516c3c4819be0fcb1ed2eb3bb3)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:52:45.728903 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a550cd-af1a-4cf9-8870-250baab6dd19 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:52:45.774390 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-92a550cd-af1a-4cf9-8870-250baab6dd19 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4d28cd516c3c4819be0fcb1ed2eb3bb3', 'request_id': 'req-92a550cd-af1a-4cf9-8870-250baab6dd19'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:52:45.779866 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-92a550cd-af1a-4cf9-8870-250baab6dd19 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4d28cd516c3c4819be0fcb1ed2eb3bb3', 'request_id': 'req-92a550cd-af1a-4cf9-8870-250baab6dd19'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:52:45.786338 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-92a550cd-af1a-4cf9-8870-250baab6dd19 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4d28cd516c3c4819be0fcb1ed2eb3bb3', 'request_id': 'req-92a550cd-af1a-4cf9-8870-250baab6dd19'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:52:45.798689 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-92a550cd-af1a-4cf9-8870-250baab6dd19 admin admin] The token cache is being invalidate because project 4d28cd516c3c4819be0fcb1ed2eb3bb3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80311) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 01 16:52:45.798980 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-92a550cd-af1a-4cf9-8870-250baab6dd19 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:52:45.799607 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-92a550cd-af1a-4cf9-8870-250baab6dd19 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:52:45.801166 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1418/2851] 10.222.0.57 () {58 vars in 1232 bytes} [Fri Jul 1 16:52:45 2022] DELETE /identity/v3/projects/4d28cd516c3c4819be0fcb1ed2eb3bb3 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 16:52:45.806224 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0849d24-d65c-467a-af5d-0ab7e73cf4cc None None] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:52:45.806745 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0849d24-d65c-467a-af5d-0ab7e73cf4cc None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:52:45.806852 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0849d24-d65c-467a-af5d-0ab7e73cf4cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:52:45.825684 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.auth.core [None req-d0849d24-d65c-467a-af5d-0ab7e73cf4cc None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80312) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:52:45.826918 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-d0849d24-d65c-467a-af5d-0ab7e73cf4cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:52:45.908613 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1434/2852] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 1 16:52:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 01 16:52:45.918280 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a993149a-d933-4c96-917a-fc3f545733fa None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:52:45.918603 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a993149a-d933-4c96-917a-fc3f545733fa None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:52:45.918889 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a993149a-d933-4c96-917a-fc3f545733fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:52:45.948600 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-a993149a-d933-4c96-917a-fc3f545733fa None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:52:45.949814 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-a993149a-d933-4c96-917a-fc3f545733fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:52:45.957937 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1419/2853] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 1 16:52:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 16:52:45.962760 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0849d24-d65c-467a-af5d-0ab7e73cf4cc None None] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:52:45.972359 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0849d24-d65c-467a-af5d-0ab7e73cf4cc None None] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:52:45.977018 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f240278-8e06-4292-bf71-a8c1f9e8fe32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:52:45.977664 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f240278-8e06-4292-bf71-a8c1f9e8fe32 admin admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:52:45.977946 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f240278-8e06-4292-bf71-a8c1f9e8fe32 admin admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:52:45.978221 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f240278-8e06-4292-bf71-a8c1f9e8fe32 admin admin] PATH_INFO: `/v3/domains` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:52:45.978981 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f240278-8e06-4292-bf71-a8c1f9e8fe32 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80312) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:52:45.979289 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f240278-8e06-4292-bf71-a8c1f9e8fe32 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:52:45.980898 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f240278-8e06-4292-bf71-a8c1f9e8fe32 admin admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:52:45.992698 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1435/2854] 10.222.0.57 () {58 vars in 1152 bytes} [Fri Jul 1 16:52:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:52:45.996995 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a993149a-d933-4c96-917a-fc3f545733fa None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:52:46.006208 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a993149a-d933-4c96-917a-fc3f545733fa None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:52:46.011778 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c3d3a3-1fab-4f36-9ee1-680d385e497e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:52:46.012430 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c3d3a3-1fab-4f36-9ee1-680d385e497e admin admin] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:52:46.012717 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c3d3a3-1fab-4f36-9ee1-680d385e497e admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:52:46.012997 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c3d3a3-1fab-4f36-9ee1-680d385e497e admin admin] PATH_INFO: `/v3/projects` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:52:46.013695 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c3d3a3-1fab-4f36-9ee1-680d385e497e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:52:46.015614 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c3d3a3-1fab-4f36-9ee1-680d385e497e admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:52:46.040750 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1420/2855] 10.222.0.57 () {60 vars in 1152 bytes} [Fri Jul 1 16:52:45 2022] POST /identity/v3/projects => generated 375 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 16:52:46.044957 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f240278-8e06-4292-bf71-a8c1f9e8fe32 admin admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:52:46.055436 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f240278-8e06-4292-bf71-a8c1f9e8fe32 admin admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:52:46.075640 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:52:46.113101 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35b5e17-ff75-4908-8b5a-cffc17d8bf1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:52:46.113763 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35b5e17-ff75-4908-8b5a-cffc17d8bf1a admin admin] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:52:46.114045 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35b5e17-ff75-4908-8b5a-cffc17d8bf1a admin admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:52:46.114319 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35b5e17-ff75-4908-8b5a-cffc17d8bf1a admin admin] PATH_INFO: `/v3/users` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:52:46.115041 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f35b5e17-ff75-4908-8b5a-cffc17d8bf1a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:52:46.117617 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f35b5e17-ff75-4908-8b5a-cffc17d8bf1a admin admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:52:46.138286 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1436/2856] 10.222.0.57 () {60 vars in 1143 bytes} [Fri Jul 1 16:52:46 2022] POST /identity/v3/users => generated 288 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 16:52:46.142616 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c3d3a3-1fab-4f36-9ee1-680d385e497e admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:52:46.152806 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c3d3a3-1fab-4f36-9ee1-680d385e497e admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:52:46.157353 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cbb07f0-1fe0-4821-9202-6a862378c64a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:52:46.158006 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbb07f0-1fe0-4821-9202-6a862378c64a admin admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:52:46.158289 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbb07f0-1fe0-4821-9202-6a862378c64a admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:52:46.158595 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbb07f0-1fe0-4821-9202-6a862378c64a admin admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:52:46.159389 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cbb07f0-1fe0-4821-9202-6a862378c64a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:52:46.160932 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cbb07f0-1fe0-4821-9202-6a862378c64a admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:52:46.171616 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1421/2857] 10.222.0.57 () {58 vars in 1121 bytes} [Fri Jul 1 16:52:46 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 16:52:46.175513 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35b5e17-ff75-4908-8b5a-cffc17d8bf1a admin admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:52:46.185932 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35b5e17-ff75-4908-8b5a-cffc17d8bf1a admin admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:52:46.190487 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51a2fea-6d76-4cee-9d85-d20a6fc5ff22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:52:46.191138 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51a2fea-6d76-4cee-9d85-d20a6fc5ff22 admin admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:52:46.191420 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51a2fea-6d76-4cee-9d85-d20a6fc5ff22 admin admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:52:46.191693 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51a2fea-6d76-4cee-9d85-d20a6fc5ff22 admin admin] PATH_INFO: `/v3/roles` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:52:46.192409 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51a2fea-6d76-4cee-9d85-d20a6fc5ff22 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:52:46.194694 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51a2fea-6d76-4cee-9d85-d20a6fc5ff22 admin admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:52:46.204570 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1437/2858] 10.222.0.57 () {58 vars in 1121 bytes} [Fri Jul 1 16:52:46 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 16:52:46.208795 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cbb07f0-1fe0-4821-9202-6a862378c64a admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:52:46.218937 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cbb07f0-1fe0-4821-9202-6a862378c64a admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:52:46.223582 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a777c64-99c6-4b26-a271-578aa28a3577 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:52:46.224453 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a777c64-99c6-4b26-a271-578aa28a3577 admin admin] REQUEST_METHOD: `PUT` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:52:46.224738 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a777c64-99c6-4b26-a271-578aa28a3577 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:52:46.225019 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a777c64-99c6-4b26-a271-578aa28a3577 admin admin] PATH_INFO: `/v3/projects/e25b0f820b1945089f2827277e3d585b/users/afab812dfa2344818a9412e4534683fb/roles/983be40bdab04c478eaa90c6b6a46b14` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:52:46.239512 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a777c64-99c6-4b26-a271-578aa28a3577 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e25b0f820b1945089f2827277e3d585b, user_id=afab812dfa2344818a9412e4534683fb, role_id=983be40bdab04c478eaa90c6b6a46b14)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:52:46.243169 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a777c64-99c6-4b26-a271-578aa28a3577 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:52:46.255832 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1422/2859] 10.222.0.57 () {60 vars in 1482 bytes} [Fri Jul 1 16:52:46 2022] PUT /identity/v3/projects/e25b0f820b1945089f2827277e3d585b/users/afab812dfa2344818a9412e4534683fb/roles/983be40bdab04c478eaa90c6b6a46b14 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 16:52:46.266912 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b56db1-bc10-463f-836e-9619f9dfcadf None None] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:52:46.267207 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b56db1-bc10-463f-836e-9619f9dfcadf None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:52:46.267491 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b56db1-bc10-463f-836e-9619f9dfcadf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:52:46.293523 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.auth.core [None req-13b56db1-bc10-463f-836e-9619f9dfcadf None None] MFA Rules not processed for user `afab812dfa2344818a9412e4534683fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80312) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:52:46.294746 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-13b56db1-bc10-463f-836e-9619f9dfcadf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:52:46.357902 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1438/2860] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 1 16:52:46 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 16:52:46.369415 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a777c64-99c6-4b26-a271-578aa28a3577 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:52:46.371162 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-0a777c64-99c6-4b26-a271-578aa28a3577 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:52:46.426781 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a777c64-99c6-4b26-a271-578aa28a3577 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:52:46.430992 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fedb21-8f00-4d38-8b91-a826c869abb1 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd028715bfea43099e7c6cde429ecf92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0788e6725a84e57a4174284c1845755', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:52:46.431644 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fedb21-8f00-4d38-8b91-a826c869abb1 service glance] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:52:46.432919 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fedb21-8f00-4d38-8b91-a826c869abb1 service glance] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:52:46.432919 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fedb21-8f00-4d38-8b91-a826c869abb1 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:52:46.440714 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01fedb21-8f00-4d38-8b91-a826c869abb1 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:52:46.442150 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01fedb21-8f00-4d38-8b91-a826c869abb1 service glance] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:52:46.451949 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1423/2861] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 1 16:52:46 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 16:52:56.355084 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=55,keystone:INSERT=3 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:52:56.448340 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:53:03.984484 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b56db1-bc10-463f-836e-9619f9dfcadf None None] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:53:04.046561 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b56db1-bc10-463f-836e-9619f9dfcadf None None] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:53:04.050992 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4e8031-a400-4fd5-ad2e-451c1fec19df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:53:04.051857 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4e8031-a400-4fd5-ad2e-451c1fec19df admin admin] REQUEST_METHOD: `DELETE` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:53:04.052134 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4e8031-a400-4fd5-ad2e-451c1fec19df admin admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:53:04.052404 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4e8031-a400-4fd5-ad2e-451c1fec19df admin admin] PATH_INFO: `/v3/users/afab812dfa2344818a9412e4534683fb` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:53:04.053654 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4e8031-a400-4fd5-ad2e-451c1fec19df admin admin] RBAC: Authorizing `identity:delete_user(user_id=afab812dfa2344818a9412e4534683fb)` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:53:04.055697 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4e8031-a400-4fd5-ad2e-451c1fec19df admin admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:53:04.102037 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-9e4e8031-a400-4fd5-ad2e-451c1fec19df admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'afab812dfa2344818a9412e4534683fb'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:53:04.113810 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-9e4e8031-a400-4fd5-ad2e-451c1fec19df admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'afab812dfa2344818a9412e4534683fb'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:53:04.120592 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-9e4e8031-a400-4fd5-ad2e-451c1fec19df admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'afab812dfa2344818a9412e4534683fb'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:53:04.121259 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-9e4e8031-a400-4fd5-ad2e-451c1fec19df admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'afab812dfa2344818a9412e4534683fb'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:53:04.121867 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-9e4e8031-a400-4fd5-ad2e-451c1fec19df admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'afab812dfa2344818a9412e4534683fb'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:53:04.134542 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1439/2862] 10.222.0.57 () {58 vars in 1223 bytes} [Fri Jul 1 16:53:03 2022] DELETE /identity/v3/users/afab812dfa2344818a9412e4534683fb => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 16:53:04.139135 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ba2c56-053e-44c1-8d73-b65481b1bc25 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:53:04.139613 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ba2c56-053e-44c1-8d73-b65481b1bc25 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:53:04.139725 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ba2c56-053e-44c1-8d73-b65481b1bc25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:53:04.166964 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-54ba2c56-053e-44c1-8d73-b65481b1bc25 None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:53:04.168181 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-54ba2c56-053e-44c1-8d73-b65481b1bc25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:53:04.227323 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1424/2863] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 1 16:53:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 16:53:04.237186 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4e8031-a400-4fd5-ad2e-451c1fec19df admin admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:53:04.238615 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-9e4e8031-a400-4fd5-ad2e-451c1fec19df admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:53:04.323029 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:53:04.331734 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4e8031-a400-4fd5-ad2e-451c1fec19df admin admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:53:04.337151 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707b7cc5-ae89-4e3d-a36e-138b74fc60d0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ed200ce29684c5f80354fa9a8638fe2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0788e6725a84e57a4174284c1845755', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:53:04.337908 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707b7cc5-ae89-4e3d-a36e-138b74fc60d0 service neutron] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:53:04.338415 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707b7cc5-ae89-4e3d-a36e-138b74fc60d0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:53:04.338694 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707b7cc5-ae89-4e3d-a36e-138b74fc60d0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:53:04.347916 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-707b7cc5-ae89-4e3d-a36e-138b74fc60d0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:53:04.349358 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-707b7cc5-ae89-4e3d-a36e-138b74fc60d0 service neutron] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:53:04.360090 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1440/2864] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 1 16:53:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 16:53:04.625351 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ba2c56-053e-44c1-8d73-b65481b1bc25 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:53:04.626797 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-54ba2c56-053e-44c1-8d73-b65481b1bc25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:53:04.638926 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ba2c56-053e-44c1-8d73-b65481b1bc25 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:53:04.643656 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da45a7e0-8dc8-413c-a155-15e438d46d66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:53:04.644521 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da45a7e0-8dc8-413c-a155-15e438d46d66 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:53:04.644803 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da45a7e0-8dc8-413c-a155-15e438d46d66 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:53:04.645079 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da45a7e0-8dc8-413c-a155-15e438d46d66 admin admin] PATH_INFO: `/v3/projects/e25b0f820b1945089f2827277e3d585b` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:53:04.646253 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da45a7e0-8dc8-413c-a155-15e438d46d66 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e25b0f820b1945089f2827277e3d585b)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:53:04.648897 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da45a7e0-8dc8-413c-a155-15e438d46d66 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:53:04.700675 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-da45a7e0-8dc8-413c-a155-15e438d46d66 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e25b0f820b1945089f2827277e3d585b', 'request_id': 'req-da45a7e0-8dc8-413c-a155-15e438d46d66'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:53:04.706552 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-da45a7e0-8dc8-413c-a155-15e438d46d66 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e25b0f820b1945089f2827277e3d585b', 'request_id': 'req-da45a7e0-8dc8-413c-a155-15e438d46d66'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:53:04.714372 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-da45a7e0-8dc8-413c-a155-15e438d46d66 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e25b0f820b1945089f2827277e3d585b', 'request_id': 'req-da45a7e0-8dc8-413c-a155-15e438d46d66'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:53:04.726997 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-da45a7e0-8dc8-413c-a155-15e438d46d66 admin admin] The token cache is being invalidate because project e25b0f820b1945089f2827277e3d585b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80311) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 01 16:53:04.727284 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-da45a7e0-8dc8-413c-a155-15e438d46d66 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:53:04.727900 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-da45a7e0-8dc8-413c-a155-15e438d46d66 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:53:04.729579 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1425/2865] 10.222.0.57 () {58 vars in 1232 bytes} [Fri Jul 1 16:53:04 2022] DELETE /identity/v3/projects/e25b0f820b1945089f2827277e3d585b => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 16:53:04.734560 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f252fa4-391e-460c-9b0d-19bd2dcc27e7 None None] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:53:04.734861 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f252fa4-391e-460c-9b0d-19bd2dcc27e7 None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:53:04.735149 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f252fa4-391e-460c-9b0d-19bd2dcc27e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:53:04.786262 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.auth.core [None req-2f252fa4-391e-460c-9b0d-19bd2dcc27e7 None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80312) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:53:04.787595 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-2f252fa4-391e-460c-9b0d-19bd2dcc27e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:53:04.856515 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1441/2866] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 1 16:53:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 16:53:04.864538 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad938be-ecba-4264-b7ed-14bd71b1251f None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:53:04.864852 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad938be-ecba-4264-b7ed-14bd71b1251f None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:53:04.865131 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad938be-ecba-4264-b7ed-14bd71b1251f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:53:04.885274 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-0ad938be-ecba-4264-b7ed-14bd71b1251f None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:53:04.886466 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-0ad938be-ecba-4264-b7ed-14bd71b1251f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:53:04.894747 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1426/2867] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 1 16:53:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 16:53:04.899698 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f252fa4-391e-460c-9b0d-19bd2dcc27e7 None None] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:53:04.909615 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f252fa4-391e-460c-9b0d-19bd2dcc27e7 None None] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:53:04.914907 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0f97d8-9161-4c8a-9c35-0a65bcf8ec24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:53:04.915592 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0f97d8-9161-4c8a-9c35-0a65bcf8ec24 admin admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:53:04.915880 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0f97d8-9161-4c8a-9c35-0a65bcf8ec24 admin admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:53:04.916162 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0f97d8-9161-4c8a-9c35-0a65bcf8ec24 admin admin] PATH_INFO: `/v3/domains` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:53:04.916916 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f0f97d8-9161-4c8a-9c35-0a65bcf8ec24 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80312) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:53:04.917228 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f0f97d8-9161-4c8a-9c35-0a65bcf8ec24 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:53:04.918964 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f0f97d8-9161-4c8a-9c35-0a65bcf8ec24 admin admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:53:04.932322 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1442/2868] 10.222.0.57 () {58 vars in 1152 bytes} [Fri Jul 1 16:53:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:53:04.936718 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad938be-ecba-4264-b7ed-14bd71b1251f None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:53:04.947331 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad938be-ecba-4264-b7ed-14bd71b1251f None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:53:04.952034 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4984bacc-ed91-4d1e-ad26-b13529e3c886 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:53:04.952699 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4984bacc-ed91-4d1e-ad26-b13529e3c886 admin admin] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:53:04.952982 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4984bacc-ed91-4d1e-ad26-b13529e3c886 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:53:04.953258 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4984bacc-ed91-4d1e-ad26-b13529e3c886 admin admin] PATH_INFO: `/v3/projects` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:53:04.953953 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4984bacc-ed91-4d1e-ad26-b13529e3c886 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:53:04.955889 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4984bacc-ed91-4d1e-ad26-b13529e3c886 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:53:04.985864 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1427/2869] 10.222.0.57 () {60 vars in 1152 bytes} [Fri Jul 1 16:53:04 2022] POST /identity/v3/projects => generated 385 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 01 16:53:04.990168 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0f97d8-9161-4c8a-9c35-0a65bcf8ec24 admin admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:53:05.001596 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0f97d8-9161-4c8a-9c35-0a65bcf8ec24 admin admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:53:05.061266 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b15c41-b920-45de-a0f9-f002045a5567 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:53:05.062033 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b15c41-b920-45de-a0f9-f002045a5567 admin admin] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:53:05.062324 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b15c41-b920-45de-a0f9-f002045a5567 admin admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:53:05.062634 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b15c41-b920-45de-a0f9-f002045a5567 admin admin] PATH_INFO: `/v3/users` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:53:05.063329 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b15c41-b920-45de-a0f9-f002045a5567 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:53:05.065268 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b15c41-b920-45de-a0f9-f002045a5567 admin admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:53:05.091815 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1443/2870] 10.222.0.57 () {60 vars in 1143 bytes} [Fri Jul 1 16:53:04 2022] POST /identity/v3/users => generated 293 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 16:53:05.096381 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4984bacc-ed91-4d1e-ad26-b13529e3c886 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:53:05.108205 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4984bacc-ed91-4d1e-ad26-b13529e3c886 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:53:05.113751 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13f8cfd-fdb3-4777-aa72-775b97a565e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:53:05.114451 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13f8cfd-fdb3-4777-aa72-775b97a565e8 admin admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:53:05.114734 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13f8cfd-fdb3-4777-aa72-775b97a565e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:53:05.115007 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13f8cfd-fdb3-4777-aa72-775b97a565e8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:53:05.116421 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e13f8cfd-fdb3-4777-aa72-775b97a565e8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:53:05.118023 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e13f8cfd-fdb3-4777-aa72-775b97a565e8 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:53:05.130038 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1428/2871] 10.222.0.57 () {58 vars in 1121 bytes} [Fri Jul 1 16:53:05 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 16:53:05.134270 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b15c41-b920-45de-a0f9-f002045a5567 admin admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:53:05.144668 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b15c41-b920-45de-a0f9-f002045a5567 admin admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:53:05.149381 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47f453b-776b-49e8-bc77-315cb96dbc1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:53:05.150050 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47f453b-776b-49e8-bc77-315cb96dbc1d admin admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:53:05.150335 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47f453b-776b-49e8-bc77-315cb96dbc1d admin admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:53:05.150643 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47f453b-776b-49e8-bc77-315cb96dbc1d admin admin] PATH_INFO: `/v3/roles` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:53:05.151386 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a47f453b-776b-49e8-bc77-315cb96dbc1d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:53:05.152984 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a47f453b-776b-49e8-bc77-315cb96dbc1d admin admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:53:05.166407 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1444/2872] 10.222.0.57 () {58 vars in 1121 bytes} [Fri Jul 1 16:53:05 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 01 16:53:05.175379 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13f8cfd-fdb3-4777-aa72-775b97a565e8 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:53:05.188752 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13f8cfd-fdb3-4777-aa72-775b97a565e8 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:53:05.194997 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0e8cae-f7fe-4fa8-8ac5-f7415f222ef8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:53:05.196008 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0e8cae-f7fe-4fa8-8ac5-f7415f222ef8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:53:05.196292 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0e8cae-f7fe-4fa8-8ac5-f7415f222ef8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:53:05.196571 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0e8cae-f7fe-4fa8-8ac5-f7415f222ef8 admin admin] PATH_INFO: `/v3/projects/793d15a9bd3944b7974490bb8a32f8f8/users/fba36a029b0144b482a4543a7e5d733e/roles/983be40bdab04c478eaa90c6b6a46b14` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:53:05.213764 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba0e8cae-f7fe-4fa8-8ac5-f7415f222ef8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=793d15a9bd3944b7974490bb8a32f8f8, user_id=fba36a029b0144b482a4543a7e5d733e, role_id=983be40bdab04c478eaa90c6b6a46b14)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:53:05.216455 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba0e8cae-f7fe-4fa8-8ac5-f7415f222ef8 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:53:05.231567 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1429/2873] 10.222.0.57 () {60 vars in 1482 bytes} [Fri Jul 1 16:53:05 2022] PUT /identity/v3/projects/793d15a9bd3944b7974490bb8a32f8f8/users/fba36a029b0144b482a4543a7e5d733e/roles/983be40bdab04c478eaa90c6b6a46b14 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 16:53:05.246620 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5107ce3f-755a-4162-acd9-ccc69b38ea3a None None] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:53:05.246917 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5107ce3f-755a-4162-acd9-ccc69b38ea3a None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:53:05.247200 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5107ce3f-755a-4162-acd9-ccc69b38ea3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:53:05.278908 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.auth.core [None req-5107ce3f-755a-4162-acd9-ccc69b38ea3a None None] MFA Rules not processed for user `fba36a029b0144b482a4543a7e5d733e`. Rule list: `[]` (Enabled: `True`). {{(pid=80312) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:53:05.280127 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-5107ce3f-755a-4162-acd9-ccc69b38ea3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:53:05.297780 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:53:05.349283 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1445/2874] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 1 16:53:05 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 16:53:05.361828 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0e8cae-f7fe-4fa8-8ac5-f7415f222ef8 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:53:05.363583 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-ba0e8cae-f7fe-4fa8-8ac5-f7415f222ef8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:53:05.378466 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:53:05.427675 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0e8cae-f7fe-4fa8-8ac5-f7415f222ef8 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:53:05.431806 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c702f0-3dbf-46b2-b755-aff4a1ec7843 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbddf971a2354bbdbb9da1d63f3abf88', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0788e6725a84e57a4174284c1845755', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:53:05.432462 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c702f0-3dbf-46b2-b755-aff4a1ec7843 service cinder] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:53:05.432753 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c702f0-3dbf-46b2-b755-aff4a1ec7843 service cinder] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:53:05.433033 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c702f0-3dbf-46b2-b755-aff4a1ec7843 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:53:05.441629 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c702f0-3dbf-46b2-b755-aff4a1ec7843 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:53:05.443217 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c702f0-3dbf-46b2-b755-aff4a1ec7843 service cinder] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:53:05.453050 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1430/2875] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 1 16:53:05 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 16:53:07.983936 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5107ce3f-755a-4162-acd9-ccc69b38ea3a None None] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:53:08.046131 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5107ce3f-755a-4162-acd9-ccc69b38ea3a None None] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:53:08.050872 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd71ec1-759a-4fe0-b5eb-ce82324ba07e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:53:08.051764 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd71ec1-759a-4fe0-b5eb-ce82324ba07e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:53:08.052079 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd71ec1-759a-4fe0-b5eb-ce82324ba07e admin admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:53:08.052384 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd71ec1-759a-4fe0-b5eb-ce82324ba07e admin admin] PATH_INFO: `/v3/users/fba36a029b0144b482a4543a7e5d733e` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:53:08.053728 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd71ec1-759a-4fe0-b5eb-ce82324ba07e admin admin] RBAC: Authorizing `identity:delete_user(user_id=fba36a029b0144b482a4543a7e5d733e)` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:53:08.055822 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd71ec1-759a-4fe0-b5eb-ce82324ba07e admin admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:53:08.102751 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-0bd71ec1-759a-4fe0-b5eb-ce82324ba07e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fba36a029b0144b482a4543a7e5d733e'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:53:08.118660 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-0bd71ec1-759a-4fe0-b5eb-ce82324ba07e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fba36a029b0144b482a4543a7e5d733e'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:53:08.127829 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-0bd71ec1-759a-4fe0-b5eb-ce82324ba07e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fba36a029b0144b482a4543a7e5d733e'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:53:08.128647 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-0bd71ec1-759a-4fe0-b5eb-ce82324ba07e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fba36a029b0144b482a4543a7e5d733e'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:53:08.129457 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-0bd71ec1-759a-4fe0-b5eb-ce82324ba07e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fba36a029b0144b482a4543a7e5d733e'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:53:08.145853 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1446/2876] 10.222.0.57 () {58 vars in 1223 bytes} [Fri Jul 1 16:53:07 2022] DELETE /identity/v3/users/fba36a029b0144b482a4543a7e5d733e => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 16:53:08.151133 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57124b82-a957-4437-9d93-244f8b70bced None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:53:08.151693 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57124b82-a957-4437-9d93-244f8b70bced None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:53:08.151693 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57124b82-a957-4437-9d93-244f8b70bced None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:53:08.169348 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-57124b82-a957-4437-9d93-244f8b70bced None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:53:08.170623 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-57124b82-a957-4437-9d93-244f8b70bced None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:53:08.229624 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1431/2877] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 1 16:53:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 16:53:08.239364 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd71ec1-759a-4fe0-b5eb-ce82324ba07e admin admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:53:08.241315 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-0bd71ec1-759a-4fe0-b5eb-ce82324ba07e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:53:08.283686 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:53:08.300276 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd71ec1-759a-4fe0-b5eb-ce82324ba07e admin admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:53:08.304075 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8483b7a4-9e62-4d00-b14d-c4465e1a0544 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ed200ce29684c5f80354fa9a8638fe2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0788e6725a84e57a4174284c1845755', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:53:08.304749 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8483b7a4-9e62-4d00-b14d-c4465e1a0544 service neutron] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:53:08.305036 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8483b7a4-9e62-4d00-b14d-c4465e1a0544 service neutron] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:53:08.305317 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8483b7a4-9e62-4d00-b14d-c4465e1a0544 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:53:08.314453 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8483b7a4-9e62-4d00-b14d-c4465e1a0544 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:53:08.315885 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8483b7a4-9e62-4d00-b14d-c4465e1a0544 service neutron] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:53:08.326404 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1447/2878] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 1 16:53:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 16:53:08.604548 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57124b82-a957-4437-9d93-244f8b70bced None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:53:08.605970 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-57124b82-a957-4437-9d93-244f8b70bced None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:53:08.619184 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57124b82-a957-4437-9d93-244f8b70bced None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:53:08.623893 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9afc6a-a13d-4b98-a07b-40db03055e49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:53:08.624760 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9afc6a-a13d-4b98-a07b-40db03055e49 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:53:08.625046 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9afc6a-a13d-4b98-a07b-40db03055e49 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:53:08.625326 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9afc6a-a13d-4b98-a07b-40db03055e49 admin admin] PATH_INFO: `/v3/projects/793d15a9bd3944b7974490bb8a32f8f8` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:53:08.626653 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a9afc6a-a13d-4b98-a07b-40db03055e49 admin admin] RBAC: Authorizing `identity:delete_project(project_id=793d15a9bd3944b7974490bb8a32f8f8)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:53:08.628518 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a9afc6a-a13d-4b98-a07b-40db03055e49 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:53:08.680109 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-6a9afc6a-a13d-4b98-a07b-40db03055e49 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '793d15a9bd3944b7974490bb8a32f8f8', 'request_id': 'req-6a9afc6a-a13d-4b98-a07b-40db03055e49'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:53:08.685475 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-6a9afc6a-a13d-4b98-a07b-40db03055e49 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '793d15a9bd3944b7974490bb8a32f8f8', 'request_id': 'req-6a9afc6a-a13d-4b98-a07b-40db03055e49'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:53:08.692082 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-6a9afc6a-a13d-4b98-a07b-40db03055e49 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '793d15a9bd3944b7974490bb8a32f8f8', 'request_id': 'req-6a9afc6a-a13d-4b98-a07b-40db03055e49'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:53:08.705029 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-6a9afc6a-a13d-4b98-a07b-40db03055e49 admin admin] The token cache is being invalidate because project 793d15a9bd3944b7974490bb8a32f8f8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80311) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 01 16:53:08.705312 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-6a9afc6a-a13d-4b98-a07b-40db03055e49 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:53:08.705926 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-6a9afc6a-a13d-4b98-a07b-40db03055e49 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:53:08.707604 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1432/2879] 10.222.0.57 () {58 vars in 1232 bytes} [Fri Jul 1 16:53:08 2022] DELETE /identity/v3/projects/793d15a9bd3944b7974490bb8a32f8f8 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 16:53:08.713086 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853e87fe-6c38-403b-a139-44444bd224e6 None None] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:53:08.713384 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853e87fe-6c38-403b-a139-44444bd224e6 None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:53:08.713670 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853e87fe-6c38-403b-a139-44444bd224e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:53:08.731897 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.auth.core [None req-853e87fe-6c38-403b-a139-44444bd224e6 None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80312) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:53:08.733088 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-853e87fe-6c38-403b-a139-44444bd224e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:53:08.791612 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1448/2880] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 1 16:53:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 16:53:08.799809 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf4c6a4-bbe8-4f7f-896e-72378823859e None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:53:08.800104 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf4c6a4-bbe8-4f7f-896e-72378823859e None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:53:08.800392 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf4c6a4-bbe8-4f7f-896e-72378823859e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:53:08.820393 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-1cf4c6a4-bbe8-4f7f-896e-72378823859e None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:53:08.821597 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-1cf4c6a4-bbe8-4f7f-896e-72378823859e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:53:08.829425 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1433/2881] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 1 16:53:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 16:53:08.836733 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853e87fe-6c38-403b-a139-44444bd224e6 None None] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:53:08.845503 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853e87fe-6c38-403b-a139-44444bd224e6 None None] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:53:08.852107 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c83bdd2-d6e8-4bfb-bf4a-ce452a34ac13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:53:08.852530 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c83bdd2-d6e8-4bfb-bf4a-ce452a34ac13 admin admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:53:08.852639 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c83bdd2-d6e8-4bfb-bf4a-ce452a34ac13 admin admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:53:08.852870 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c83bdd2-d6e8-4bfb-bf4a-ce452a34ac13 admin admin] PATH_INFO: `/v3/domains` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:53:08.853592 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c83bdd2-d6e8-4bfb-bf4a-ce452a34ac13 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80312) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 16:53:08.853908 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c83bdd2-d6e8-4bfb-bf4a-ce452a34ac13 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:53:08.855566 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c83bdd2-d6e8-4bfb-bf4a-ce452a34ac13 admin admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:53:08.868096 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1449/2882] 10.222.0.57 () {58 vars in 1152 bytes} [Fri Jul 1 16:53:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 16:53:08.872630 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf4c6a4-bbe8-4f7f-896e-72378823859e None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:53:08.882117 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf4c6a4-bbe8-4f7f-896e-72378823859e None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:53:08.886791 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0c0d2e-9486-40f7-bf7b-b20817416656 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:53:08.887497 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0c0d2e-9486-40f7-bf7b-b20817416656 admin admin] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:53:08.887786 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0c0d2e-9486-40f7-bf7b-b20817416656 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:53:08.888066 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0c0d2e-9486-40f7-bf7b-b20817416656 admin admin] PATH_INFO: `/v3/projects` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:53:08.888766 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb0c0d2e-9486-40f7-bf7b-b20817416656 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:53:08.890684 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb0c0d2e-9486-40f7-bf7b-b20817416656 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:53:08.916035 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1434/2883] 10.222.0.57 () {60 vars in 1152 bytes} [Fri Jul 1 16:53:08 2022] POST /identity/v3/projects => generated 383 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 16:53:08.920192 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c83bdd2-d6e8-4bfb-bf4a-ce452a34ac13 admin admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:53:08.930983 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c83bdd2-d6e8-4bfb-bf4a-ce452a34ac13 admin admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:53:08.985290 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e0b303-cbcd-4d7c-b5c0-c134785ea6f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:53:08.985957 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e0b303-cbcd-4d7c-b5c0-c134785ea6f8 admin admin] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:53:08.986240 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e0b303-cbcd-4d7c-b5c0-c134785ea6f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:53:08.986543 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e0b303-cbcd-4d7c-b5c0-c134785ea6f8 admin admin] PATH_INFO: `/v3/users` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:53:08.987243 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e0b303-cbcd-4d7c-b5c0-c134785ea6f8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:53:08.989159 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e0b303-cbcd-4d7c-b5c0-c134785ea6f8 admin admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:53:09.010888 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1450/2884] 10.222.0.57 () {60 vars in 1143 bytes} [Fri Jul 1 16:53:08 2022] POST /identity/v3/users => generated 292 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 16:53:09.015034 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0c0d2e-9486-40f7-bf7b-b20817416656 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:53:09.026276 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0c0d2e-9486-40f7-bf7b-b20817416656 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:53:09.031424 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9fad61-1853-43f5-a8fc-2b330b9a6634 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:53:09.032106 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9fad61-1853-43f5-a8fc-2b330b9a6634 admin admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:53:09.032375 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9fad61-1853-43f5-a8fc-2b330b9a6634 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:53:09.032651 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9fad61-1853-43f5-a8fc-2b330b9a6634 admin admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:53:09.033384 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e9fad61-1853-43f5-a8fc-2b330b9a6634 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:53:09.035018 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e9fad61-1853-43f5-a8fc-2b330b9a6634 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:53:09.041244 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:53:09.047062 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1435/2885] 10.222.0.57 () {58 vars in 1121 bytes} [Fri Jul 1 16:53:09 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 16:53:09.051271 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e0b303-cbcd-4d7c-b5c0-c134785ea6f8 admin admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:53:09.062405 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e0b303-cbcd-4d7c-b5c0-c134785ea6f8 admin admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:53:09.067088 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa45aae0-cec9-4700-884f-db783187eacb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:53:09.067748 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa45aae0-cec9-4700-884f-db783187eacb admin admin] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:53:09.068029 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa45aae0-cec9-4700-884f-db783187eacb admin admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:53:09.068305 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa45aae0-cec9-4700-884f-db783187eacb admin admin] PATH_INFO: `/v3/roles` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:53:09.069189 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa45aae0-cec9-4700-884f-db783187eacb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:53:09.070829 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa45aae0-cec9-4700-884f-db783187eacb admin admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:53:09.081002 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1451/2886] 10.222.0.57 () {58 vars in 1121 bytes} [Fri Jul 1 16:53:09 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 16:53:09.085596 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9fad61-1853-43f5-a8fc-2b330b9a6634 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:53:09.095562 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9fad61-1853-43f5-a8fc-2b330b9a6634 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:53:09.100893 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8de906-8094-45a9-aa78-95c311a0d473 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:53:09.101837 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8de906-8094-45a9-aa78-95c311a0d473 admin admin] REQUEST_METHOD: `PUT` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:53:09.102119 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8de906-8094-45a9-aa78-95c311a0d473 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:53:09.102428 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8de906-8094-45a9-aa78-95c311a0d473 admin admin] PATH_INFO: `/v3/projects/0879ab147a0b4fbeb3ec90fdfdb53f24/users/520e0eae82a64ed98a6c0cfbd19cdbb4/roles/983be40bdab04c478eaa90c6b6a46b14` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:53:09.117731 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad8de906-8094-45a9-aa78-95c311a0d473 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0879ab147a0b4fbeb3ec90fdfdb53f24, user_id=520e0eae82a64ed98a6c0cfbd19cdbb4, role_id=983be40bdab04c478eaa90c6b6a46b14)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:53:09.120398 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad8de906-8094-45a9-aa78-95c311a0d473 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:53:09.133069 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1436/2887] 10.222.0.57 () {60 vars in 1482 bytes} [Fri Jul 1 16:53:09 2022] PUT /identity/v3/projects/0879ab147a0b4fbeb3ec90fdfdb53f24/users/520e0eae82a64ed98a6c0cfbd19cdbb4/roles/983be40bdab04c478eaa90c6b6a46b14 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 16:53:09.138969 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443e533b-fcd9-483a-8050-6fc3fc1fedd4 None None] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:53:09.139454 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443e533b-fcd9-483a-8050-6fc3fc1fedd4 None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:53:09.139668 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443e533b-fcd9-483a-8050-6fc3fc1fedd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:53:09.157447 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.auth.core [None req-443e533b-fcd9-483a-8050-6fc3fc1fedd4 None None] MFA Rules not processed for user `9996f66c673f4673ab2342b66da9acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80312) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:53:09.158710 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-443e533b-fcd9-483a-8050-6fc3fc1fedd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:53:09.200981 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:53:09.219712 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1452/2888] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 1 16:53:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 16:53:09.229821 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8de906-8094-45a9-aa78-95c311a0d473 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:53:09.231735 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-ad8de906-8094-45a9-aa78-95c311a0d473 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:53:09.290027 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8de906-8094-45a9-aa78-95c311a0d473 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:53:09.293746 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1ae1cc-314c-4ff8-8802-e99eee815c30 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ed200ce29684c5f80354fa9a8638fe2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0788e6725a84e57a4174284c1845755', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:53:09.294450 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1ae1cc-314c-4ff8-8802-e99eee815c30 service neutron] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:53:09.294724 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1ae1cc-314c-4ff8-8802-e99eee815c30 service neutron] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:53:09.295061 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1ae1cc-314c-4ff8-8802-e99eee815c30 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:53:09.304759 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb1ae1cc-314c-4ff8-8802-e99eee815c30 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:53:09.306209 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb1ae1cc-314c-4ff8-8802-e99eee815c30 service neutron] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:53:09.315840 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1437/2889] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 1 16:53:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 16:53:09.960226 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef57ed60-7997-452b-a1db-7ddc8b12fa29 None None] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:53:09.961534 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef57ed60-7997-452b-a1db-7ddc8b12fa29 None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:53:09.961918 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef57ed60-7997-452b-a1db-7ddc8b12fa29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:53:09.988637 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.auth.core [None req-ef57ed60-7997-452b-a1db-7ddc8b12fa29 None None] MFA Rules not processed for user `520e0eae82a64ed98a6c0cfbd19cdbb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80312) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:53:09.989955 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-ef57ed60-7997-452b-a1db-7ddc8b12fa29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:53:10.058755 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1453/2890] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 1 16:53:09 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 16:53:10.070167 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1ae1cc-314c-4ff8-8802-e99eee815c30 service neutron] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:53:10.071416 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-eb1ae1cc-314c-4ff8-8802-e99eee815c30 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:53:10.133927 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1ae1cc-314c-4ff8-8802-e99eee815c30 service neutron] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:53:10.138678 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d1d5ae-f290-4835-83f6-2f61201a4625 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbddf971a2354bbdbb9da1d63f3abf88', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0788e6725a84e57a4174284c1845755', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:53:10.139375 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d1d5ae-f290-4835-83f6-2f61201a4625 service cinder] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:53:10.139643 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d1d5ae-f290-4835-83f6-2f61201a4625 service cinder] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:53:10.140015 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d1d5ae-f290-4835-83f6-2f61201a4625 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:53:10.149319 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d1d5ae-f290-4835-83f6-2f61201a4625 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:53:10.150778 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d1d5ae-f290-4835-83f6-2f61201a4625 service cinder] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:53:10.159548 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1438/2891] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 1 16:53:10 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 16:53:11.500654 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9e9290-1925-4ab9-a2fa-da711e87e883 None None] REQUEST_METHOD: `POST` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:53:11.501183 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9e9290-1925-4ab9-a2fa-da711e87e883 None None] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:53:11.501298 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9e9290-1925-4ab9-a2fa-da711e87e883 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:53:11.519604 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.auth.core [None req-1e9e9290-1925-4ab9-a2fa-da711e87e883 None None] MFA Rules not processed for user `520e0eae82a64ed98a6c0cfbd19cdbb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80312) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:53:11.520845 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-1e9e9290-1925-4ab9-a2fa-da711e87e883 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:53:11.530813 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1454/2892] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 1 16:53:11 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 16:53:11.778368 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d1d5ae-f290-4835-83f6-2f61201a4625 service cinder] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:53:11.788287 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d1d5ae-f290-4835-83f6-2f61201a4625 service cinder] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:53:11.791901 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f9b531-9541-4294-85a8-8e5de7fb30e3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ed200ce29684c5f80354fa9a8638fe2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0788e6725a84e57a4174284c1845755', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:53:11.792529 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f9b531-9541-4294-85a8-8e5de7fb30e3 service neutron] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:53:11.792811 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f9b531-9541-4294-85a8-8e5de7fb30e3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:53:11.793089 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f9b531-9541-4294-85a8-8e5de7fb30e3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:53:11.794842 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-b0f9b531-9541-4294-85a8-8e5de7fb30e3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:53:11.804657 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f9b531-9541-4294-85a8-8e5de7fb30e3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:53:11.806063 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f9b531-9541-4294-85a8-8e5de7fb30e3 service neutron] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:53:11.813940 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1439/2893] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 1 16:53:11 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 16:53:19.970258 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9e9290-1925-4ab9-a2fa-da711e87e883 None None] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:53:19.971816 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-1e9e9290-1925-4ab9-a2fa-da711e87e883 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:53:20.058566 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9e9290-1925-4ab9-a2fa-da711e87e883 None None] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:53:20.064091 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf4d4ff-961f-443d-9be6-b9520970f454 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a50f745b31941b7a7265c0fc852f48e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0788e6725a84e57a4174284c1845755', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:53:20.064739 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf4d4ff-961f-443d-9be6-b9520970f454 service nova] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:53:20.065037 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf4d4ff-961f-443d-9be6-b9520970f454 service nova] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:53:20.065318 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf4d4ff-961f-443d-9be6-b9520970f454 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:53:20.067120 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-dcf4d4ff-961f-443d-9be6-b9520970f454 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:53:20.080232 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf4d4ff-961f-443d-9be6-b9520970f454 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:53:20.083630 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf4d4ff-961f-443d-9be6-b9520970f454 service nova] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:53:20.097429 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1455/2894] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 1 16:53:19 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 16:53:21.810471 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=68,keystone:INSERT=1 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:53:30.092853 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=78 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:54:14.759312 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff74d5a-a7f5-4ac3-8fda-c679b8b2264e None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:54:14.760170 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff74d5a-a7f5-4ac3-8fda-c679b8b2264e None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:54:14.760170 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff74d5a-a7f5-4ac3-8fda-c679b8b2264e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:54:14.778888 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-7ff74d5a-a7f5-4ac3-8fda-c679b8b2264e None None] MFA Rules not processed for user `520e0eae82a64ed98a6c0cfbd19cdbb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 16:54:14.780101 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-7ff74d5a-a7f5-4ac3-8fda-c679b8b2264e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:54:14.787012 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1440/2895] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 1 16:54:14 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 16:54:15.018939 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf4d4ff-961f-443d-9be6-b9520970f454 service nova] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:54:15.028669 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf4d4ff-961f-443d-9be6-b9520970f454 service nova] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:54:15.032261 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633a5c58-4523-4150-a316-d4a0099aa935 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ed200ce29684c5f80354fa9a8638fe2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0788e6725a84e57a4174284c1845755', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:54:15.032921 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633a5c58-4523-4150-a316-d4a0099aa935 service neutron] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:54:15.033199 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633a5c58-4523-4150-a316-d4a0099aa935 service neutron] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:54:15.033471 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633a5c58-4523-4150-a316-d4a0099aa935 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:54:15.035225 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-633a5c58-4523-4150-a316-d4a0099aa935 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:54:15.045052 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-633a5c58-4523-4150-a316-d4a0099aa935 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:54:15.046490 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-633a5c58-4523-4150-a316-d4a0099aa935 service neutron] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:54:15.054424 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1456/2896] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 1 16:54:15 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 16:54:24.772686 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=3 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:54:25.050737 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=6 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:54:30.665978 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff74d5a-a7f5-4ac3-8fda-c679b8b2264e None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:54:30.667435 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-7ff74d5a-a7f5-4ac3-8fda-c679b8b2264e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:54:30.757861 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff74d5a-a7f5-4ac3-8fda-c679b8b2264e None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:54:30.763834 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd23c610-1c13-49a9-95a4-c704dad5ef95 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc38af18dec4e9b8143df23af6aad43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0788e6725a84e57a4174284c1845755', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:54:30.764479 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd23c610-1c13-49a9-95a4-c704dad5ef95 service placement] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:54:30.764767 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd23c610-1c13-49a9-95a4-c704dad5ef95 service placement] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:54:30.765047 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd23c610-1c13-49a9-95a4-c704dad5ef95 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:54:30.766845 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-bd23c610-1c13-49a9-95a4-c704dad5ef95 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:54:30.779029 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd23c610-1c13-49a9-95a4-c704dad5ef95 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:54:30.780611 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd23c610-1c13-49a9-95a4-c704dad5ef95 service placement] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:54:30.791136 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1441/2897] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 1 16:54:30 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 16:54:40.786242 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=41 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:55:12.300319 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633a5c58-4523-4150-a316-d4a0099aa935 service neutron] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:55:12.312567 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633a5c58-4523-4150-a316-d4a0099aa935 service neutron] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:55:12.317137 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7b65ad-f321-4443-96bf-cbf91c0992ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:55:12.318081 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7b65ad-f321-4443-96bf-cbf91c0992ec admin admin] REQUEST_METHOD: `DELETE` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:55:12.318388 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7b65ad-f321-4443-96bf-cbf91c0992ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:55:12.318667 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7b65ad-f321-4443-96bf-cbf91c0992ec admin admin] PATH_INFO: `/v3/users/520e0eae82a64ed98a6c0cfbd19cdbb4` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:55:12.319876 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d7b65ad-f321-4443-96bf-cbf91c0992ec admin admin] RBAC: Authorizing `identity:delete_user(user_id=520e0eae82a64ed98a6c0cfbd19cdbb4)` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:55:12.322662 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d7b65ad-f321-4443-96bf-cbf91c0992ec admin admin] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:55:12.375451 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-1d7b65ad-f321-4443-96bf-cbf91c0992ec admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '520e0eae82a64ed98a6c0cfbd19cdbb4'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:55:12.389339 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-1d7b65ad-f321-4443-96bf-cbf91c0992ec admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '520e0eae82a64ed98a6c0cfbd19cdbb4'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:55:12.396585 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-1d7b65ad-f321-4443-96bf-cbf91c0992ec admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '520e0eae82a64ed98a6c0cfbd19cdbb4'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:55:12.397335 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-1d7b65ad-f321-4443-96bf-cbf91c0992ec admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '520e0eae82a64ed98a6c0cfbd19cdbb4'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:55:12.397950 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.notifications [None req-1d7b65ad-f321-4443-96bf-cbf91c0992ec admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '520e0eae82a64ed98a6c0cfbd19cdbb4'} {{(pid=80312) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:55:12.412249 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1457/2898] 10.222.0.57 () {58 vars in 1223 bytes} [Fri Jul 1 16:55:12 2022] DELETE /identity/v3/users/520e0eae82a64ed98a6c0cfbd19cdbb4 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 16:55:12.596582 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd23c610-1c13-49a9-95a4-c704dad5ef95 service placement] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:55:12.608896 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd23c610-1c13-49a9-95a4-c704dad5ef95 service placement] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:55:12.670457 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66c67ae-ad9c-43c7-bc8c-5f12d913793f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9996f66c673f4673ab2342b66da9acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62a34c02685249b399e8facc39cc11f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:55:12.671288 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66c67ae-ad9c-43c7-bc8c-5f12d913793f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:55:12.671580 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66c67ae-ad9c-43c7-bc8c-5f12d913793f admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:55:12.671862 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66c67ae-ad9c-43c7-bc8c-5f12d913793f admin admin] PATH_INFO: `/v3/projects/0879ab147a0b4fbeb3ec90fdfdb53f24` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:55:12.673160 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66c67ae-ad9c-43c7-bc8c-5f12d913793f admin admin] RBAC: Authorizing `identity:delete_project(project_id=0879ab147a0b4fbeb3ec90fdfdb53f24)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:55:12.675241 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66c67ae-ad9c-43c7-bc8c-5f12d913793f admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:55:12.724739 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-e66c67ae-ad9c-43c7-bc8c-5f12d913793f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0879ab147a0b4fbeb3ec90fdfdb53f24', 'request_id': 'req-e66c67ae-ad9c-43c7-bc8c-5f12d913793f'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:55:12.731197 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-e66c67ae-ad9c-43c7-bc8c-5f12d913793f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0879ab147a0b4fbeb3ec90fdfdb53f24', 'request_id': 'req-e66c67ae-ad9c-43c7-bc8c-5f12d913793f'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:55:12.737416 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-e66c67ae-ad9c-43c7-bc8c-5f12d913793f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0879ab147a0b4fbeb3ec90fdfdb53f24', 'request_id': 'req-e66c67ae-ad9c-43c7-bc8c-5f12d913793f'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:55:12.751971 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-e66c67ae-ad9c-43c7-bc8c-5f12d913793f admin admin] The token cache is being invalidate because project 0879ab147a0b4fbeb3ec90fdfdb53f24 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80311) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 01 16:55:12.751971 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-e66c67ae-ad9c-43c7-bc8c-5f12d913793f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:55:12.752509 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-e66c67ae-ad9c-43c7-bc8c-5f12d913793f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 16:55:12.754114 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1442/2899] 10.222.0.57 () {58 vars in 1232 bytes} [Fri Jul 1 16:55:12 2022] DELETE /identity/v3/projects/0879ab147a0b4fbeb3ec90fdfdb53f24 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 16:55:22.408379 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=29,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:55:22.748022 dv-cinder-797608-41-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=51,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 16:59:38.669307 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7b65ad-f321-4443-96bf-cbf91c0992ec admin admin] Authenticating user token {{(pid=80312) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 16:59:38.670802 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-1d7b65ad-f321-4443-96bf-cbf91c0992ec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:59:38.743319 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7b65ad-f321-4443-96bf-cbf91c0992ec admin admin] Validating token access rules against request {{(pid=80312) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 16:59:38.802933 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0000b5-a955-481d-ba3f-10253bd205d6 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc38af18dec4e9b8143df23af6aad43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0788e6725a84e57a4174284c1845755', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80312) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 16:59:38.803580 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0000b5-a955-481d-ba3f-10253bd205d6 service placement] REQUEST_METHOD: `GET` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 16:59:38.803866 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0000b5-a955-481d-ba3f-10253bd205d6 service placement] SCRIPT_NAME: `/identity` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 16:59:38.804137 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0000b5-a955-481d-ba3f-10253bd205d6 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80312) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 16:59:38.805904 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.fernet_utils [None req-9c0000b5-a955-481d-ba3f-10253bd205d6 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80312) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 16:59:38.817430 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c0000b5-a955-481d-ba3f-10253bd205d6 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 16:59:38.819030 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c0000b5-a955-481d-ba3f-10253bd205d6 service placement] RBAC: Authorization granted {{(pid=80312) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 16:59:38.828733 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: [pid: 80312|app: 0|req: 1458/2900] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 1 16:59:38 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 16:59:48.824552 dv-cinder-797608-41-iscsi devstack@keystone.service[80312]: DEBUG dbcounter [-] [80312] Writing DB stats keystone:SELECT=58 {{(pid=80312) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}